By type
[2021-05-28T00:05:48+00:00] INFO: Started chef-zero at chefzero://localhost:1 with repository at /var/lib/jetty/chef-solo One version per cookbook [2021-05-28T00:05:48+00:00] INFO: Started chef-zero at chefzero://localhost:1 with repository at /var/lib/jetty/chef-solo One version per cookbook [2021-05-28T00:05:48+00:00] INFO: Forking chef instance to converge... [2021-05-28T00:05:48+00:00] INFO: Forking chef instance to converge... Starting Chef Client, version 13.8.5[0m [2021-05-28T00:05:48+00:00] INFO: *** Chef 13.8.5 *** [2021-05-28T00:05:48+00:00] INFO: *** Chef 13.8.5 *** [2021-05-28T00:05:48+00:00] INFO: Platform: x86_64-linux-gnu [2021-05-28T00:05:48+00:00] INFO: Platform: x86_64-linux-gnu [2021-05-28T00:05:48+00:00] INFO: Chef-client pid: 22 [2021-05-28T00:05:48+00:00] INFO: Chef-client pid: 22 [2021-05-28T00:05:48+00:00] INFO: The plugin path /etc/chef/ohai/plugins does not exist. Skipping... [2021-05-28T00:05:48+00:00] INFO: The plugin path /etc/chef/ohai/plugins does not exist. Skipping... [2021-05-28T00:05:50+00:00] WARN: Plugin Network: unable to detect ipaddress [2021-05-28T00:05:50+00:00] WARN: Plugin Network: unable to detect ipaddress [2021-05-28T00:05:50+00:00] INFO: Setting the run_list to ["role[catalog-be]"] from CLI options [2021-05-28T00:05:50+00:00] INFO: Setting the run_list to ["role[catalog-be]"] from CLI options [2021-05-28T00:05:50+00:00] INFO: Run List is [role[catalog-be]] [2021-05-28T00:05:50+00:00] INFO: Run List is [role[catalog-be]] [2021-05-28T00:05:50+00:00] INFO: Run List expands to [sdc-catalog-be::BE_1_cleanup_jettydir, sdc-catalog-be::BE_2_setup_configuration, sdc-catalog-be::BE_3_locate_keystore, sdc-catalog-be::BE_4_jetty_Modules, sdc-catalog-be::BE_6_setup_portal_and_key_properties, sdc-catalog-be::BE_7_logback, sdc-catalog-be::BE_8_errors_config, sdc-catalog-be::BE_9_prepareProbeFile, sdc-catalog-be::BE_10_setup_cadi] [2021-05-28T00:05:50+00:00] INFO: Run List expands to [sdc-catalog-be::BE_1_cleanup_jettydir, sdc-catalog-be::BE_2_setup_configuration, sdc-catalog-be::BE_3_locate_keystore, sdc-catalog-be::BE_4_jetty_Modules, sdc-catalog-be::BE_6_setup_portal_and_key_properties, sdc-catalog-be::BE_7_logback, sdc-catalog-be::BE_8_errors_config, sdc-catalog-be::BE_9_prepareProbeFile, sdc-catalog-be::BE_10_setup_cadi] [2021-05-28T00:05:50+00:00] INFO: Starting Chef Run for dev-sdc-be-7697c4f957-9fbvk [2021-05-28T00:05:50+00:00] INFO: Starting Chef Run for dev-sdc-be-7697c4f957-9fbvk [2021-05-28T00:05:50+00:00] INFO: Running start handlers [2021-05-28T00:05:50+00:00] INFO: Running start handlers [2021-05-28T00:05:50+00:00] INFO: Start handlers complete. [2021-05-28T00:05:50+00:00] INFO: Start handlers complete. resolving cookbooks for run list: ["sdc-catalog-be::BE_1_cleanup_jettydir", "sdc-catalog-be::BE_2_setup_configuration", "sdc-catalog-be::BE_3_locate_keystore", "sdc-catalog-be::BE_4_jetty_Modules", "sdc-catalog-be::BE_6_setup_portal_and_key_properties", "sdc-catalog-be::BE_7_logback", "sdc-catalog-be::BE_8_errors_config", "sdc-catalog-be::BE_9_prepareProbeFile", "sdc-catalog-be::BE_10_setup_cadi"][0m [2021-05-28T00:05:51+00:00] INFO: Loading cookbooks [sdc-catalog-be@0.0.0] [2021-05-28T00:05:51+00:00] INFO: Loading cookbooks [sdc-catalog-be@0.0.0] Synchronizing Cookbooks:[0m [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-onboarding-configuration.yaml.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-onboarding-configuration.yaml.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-key.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-key.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/attributes/default.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/attributes/default.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/http-ini.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/http-ini.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-portal.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-portal.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-distribution-engine-configuration.yaml.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-distribution-engine-configuration.yaml.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-configuration.yaml.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-configuration.yaml.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-janusgraph.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/BE-janusgraph.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/cadi.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/cadi.properties.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/ready-probe.sh.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/ready-probe.sh.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/https-ini.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/https-ini.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/cadi_truststore.jks in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/cadi_truststore.jks in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/Artifact-Generator.properties in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/Artifact-Generator.properties in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/ssl-ini.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/ecomp-error-configuration.yaml in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/ecomp-error-configuration.yaml in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/templates/default/ssl-ini.erb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/cadi.properties in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/cadi.properties in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/error-configuration.yaml in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/error-configuration.yaml in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_10_setup_cadi.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/keyfile in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/keyfile in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_10_setup_cadi.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/org.onap.sdc.p12 in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/org.onap.sdc.p12 in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_1_cleanup_jettydir.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_1_cleanup_jettydir.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/org.onap.sdc.trust.jks in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/org.onap.sdc.trust.jks in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/logback.xml in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/files/default/logback.xml in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_2_setup_configuration.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_2_setup_configuration.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_3_locate_keystore.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_3_locate_keystore.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_6_setup_portal_and_key_properties.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_6_setup_portal_and_key_properties.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_4_jetty_Modules.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_4_jetty_Modules.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_7_logback.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_7_logback.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_8_errors_config.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_8_errors_config.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_5_setup_elasticsearch.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_9_prepareProbeFile.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_9_prepareProbeFile.rb in the cache. [2021-05-28T00:05:51+00:00] INFO: Storing updated cookbooks/sdc-catalog-be/recipes/BE_5_setup_elasticsearch.rb in the cache. - sdc-catalog-be (0.0.0)[0m Installing Cookbook Gems:[0m Compiling Cookbooks...[0m Converging 23 resources[0m Recipe: sdc-catalog-be::BE_1_cleanup_jettydir[0m * directory[BE_tempdir_creation] action create[2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] created directory /var/lib/jetty/temp [2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] created directory /var/lib/jetty/temp [32m- create new directory /var/lib/jetty/temp[0m[2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] mode changed to 755 [2021-05-28T00:05:51+00:00] INFO: directory[BE_tempdir_creation] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * directory[BE_create_config_dir] action create[2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] created directory /var/lib/jetty/config [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] created directory /var/lib/jetty/config [32m- create new directory /var/lib/jetty/config[0m[2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] mode changed to 755 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_config_dir] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * directory[BE_create_catalog-be] action create[2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] created directory /var/lib/jetty/config/catalog-be [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] created directory /var/lib/jetty/config/catalog-be [32m- create new directory /var/lib/jetty/config/catalog-be[0m[2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] mode changed to 755 [2021-05-28T00:05:51+00:00] INFO: directory[BE_create_catalog-be] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0mRecipe: sdc-catalog-be::BE_2_setup_configuration[0m * template[janusgraph.properties] action create_if_missing[2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] created file /var/lib/jetty/config/catalog-be/janusgraph.properties [2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] created file /var/lib/jetty/config/catalog-be/janusgraph.properties [32m- create new file /var/lib/jetty/config/catalog-be/janusgraph.properties[0m[2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] updated file contents /var/lib/jetty/config/catalog-be/janusgraph.properties [2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] updated file contents /var/lib/jetty/config/catalog-be/janusgraph.properties [32m- update content in file /var/lib/jetty/config/catalog-be/janusgraph.properties from none to 18f7e5[0m [37m--- /var/lib/jetty/config/catalog-be/janusgraph.properties 2021-05-28 00:05:51.184662975 +0000[0m [37m+++ /var/lib/jetty/config/catalog-be/.chef-janusgraph20210528-22-9pqyqm.properties 2021-05-28 00:05:51.184662975 +0000[0m [37m@@ -1,27 +1,54 @@[0m [37m+storage.backend=cql[0m [37m+storage.hostname=cassandra.onap[0m [37m+storage.port=9042[0m [37m+storage.username=asdc_user[0m [37m+storage.password=Aa1234%^![0m [37m+storage.connection-timeout=10000[0m [37m+storage.cql.keyspace=sdctitan[0m [37m+[0m [37m+storage.cql.ssl.enabled=false[0m [37m+storage.cql.ssl.truststore.location=[0m [37m+storage.cql.ssl.truststore.password=Aa123456[0m [37m+[0m [37m+storage.cql.read-consistency-level=LOCAL_QUORUM[0m [37m+storage.cql.write-consistency-level=LOCAL_QUORUM[0m [37m+storage.cql.replication-strategy-class=NetworkTopologyStrategy[0m [37m+storage.cql.replication-strategy-options=Pod,3[0m [37m+storage.cql.local-datacenter=Pod[0m [37m+[0m [37m+[0m [37m+cache.db-cache = false[0m [37m+cache.db-cache-clean-wait = 20[0m [37m+cache.db-cache-time = 180000[0m [37m+cache.db-cache-size = 0.5[0m [37m+[0m [37m+cache.tx-cache-size = 1000000[0m [37m+[0m [37m+storage.lock.retries=5[0m[2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] mode changed to 644 [2021-05-28T00:05:51+00:00] INFO: template[janusgraph.properties] mode changed to 644 [32m- change mode from '' to '0644'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * template[catalog-be-config] action create_if_missing[2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] created file /var/lib/jetty/config/catalog-be/configuration.yaml [2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] created file /var/lib/jetty/config/catalog-be/configuration.yaml [32m- create new file /var/lib/jetty/config/catalog-be/configuration.yaml[0m[2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] updated file contents /var/lib/jetty/config/catalog-be/configuration.yaml [2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] updated file contents /var/lib/jetty/config/catalog-be/configuration.yaml [32m- update content in file /var/lib/jetty/config/catalog-be/configuration.yaml from none to fbe78e[0m [37m--- /var/lib/jetty/config/catalog-be/configuration.yaml 2021-05-28 00:05:51.196663759 +0000[0m [37m+++ /var/lib/jetty/config/catalog-be/.chef-configuration20210528-22-yekzfp.yaml 2021-05-28 00:05:51.192663497 +0000[0m [37m@@ -1,1243 +1,2486 @@[0m [37m+identificationHeaderFields:[0m [37m+ - HTTP_IV_USER[0m [37m+ - HTTP_CSP_FIRSTNAME[0m [37m+ - HTTP_CSP_LASTNAME[0m [37m+ - HTTP_IV_REMOTE_ADDRESS[0m [37m+ - HTTP_CSP_WSTYPE[0m [37m+[0m [37m+# catalog backend hostname[0m [37m+beFqdn: sdc-be.onap[0m [37m+[0m [37m+# catalog backend http port[0m [37m+beHttpPort: 8080[0m [37m+[0m [37m+# catalog backend http context[0m [37m+beContext: /sdc/rest/config/get[0m [37m+[0m [37m+# catalog backend protocol[0m [37m+beProtocol: https[0m [37m+[0m [37m+# catalog backend ssl port[0m [37m+beSslPort: 8443[0m [37m+version: 1.1.0[0m [37m+released: 2012-11-30[0m [37m+toscaConformanceLevel: 11.0[0m [37m+minToscaConformanceLevel: 3.0[0m [37m+[0m [37m+janusGraphCfgFile: /var/lib/jetty/config/catalog-be/janusgraph.properties[0m [37m+janusGraphInMemoryGraph: false[0m [37m+janusGraphLockTimeout: 1800[0m [37m+# The interval to try and reconnect to JanusGraph DB when it is down during ASDC startup:[0m [37m+janusGraphReconnectIntervalInSeconds: 3[0m [37m+[0m [37m+# The read timeout towards JanusGraph DB when health check is invoked:[0m [37m+janusGraphHealthCheckReadTimeout: 1[0m [37m+[0m [37m+# The interval to try and reconnect to Elasticsearch when it is down during ASDC startup:[0m [37m+[0m [37m+uebHealthCheckReconnectIntervalInSeconds: 15[0m [37m+uebHealthCheckReadTimeout: 4[0m [37m+[0m [37m+# Protocols[0m [37m+protocols:[0m [37m+ - http[0m [37m+ - https[0m [37m+[0m [37m+# Default imports[0m [37m+defaultImports:[0m [37m+ - nodes:[0m [37m+ file: nodes.yml[0m [37m+ - datatypes:[0m [37m+ file: data.yml[0m [37m+ - capabilities:[0m [37m+ file: capabilities.yml[0m [37m+ - relationships:[0m [37m+ file: relationships.yml[0m [37m+ - groups:[0m [37m+ file: groups.yml[0m [37m+ - policies:[0m [37m+ file: policies.yml[0m [37m+ - annotations:[0m [37m+ file: annotations.yml[0m [37m+[0m [37m+# Global CSAR Import Files[0m [37m+globalCsarImports:[0m [37m+ - annotations.yml[0m [37m+ - artifacts.yml[0m [37m+ - capabilities.yml[0m [37m+ - data.yml[0m [37m+ - groups.yml[0m [37m+ - interfaces.yml[0m [37m+ - nodes.yml[0m [37m+ - policies.yml[0m [37m+ - relationships.yml[0m [37m+[0m [37m+# Users[0m [37m+users:[0m [37m+ tom: passwd[0m [37m+ bob: passwd[0m [37m+[0m [37m+# access restriction[0m [37m+authCookie:[0m [37m+ securityKey: "sdcaccessrestrictionsecureykey"[0m [37m+ maxSessionTimeOut: 86400000[0m [37m+ sessionIdleTimeOut: 3600000[0m [37m+ cookieName: "AuthenticationCookie"[0m [37m+ path: /[0m [37m+ domain: ""[0m [37m+ isHttpOnly: true[0m [37m+ # redirect variable name from portal.properties file[0m [37m+ redirectURL: ""[0m [37m+ excludedUrls: ['/.*'][0m [37m+ onboardingExcludedUrls: ['/.*'][0m [37m+[0m [37m+basicAuth:[0m [37m+ enabled: false[0m [37m+ userName: testName[0m [37m+ userPass: testPass[0m [37m+ excludedUrls: "/sdc2/rest/healthCheck"[0m [37m+[0m [37m+cassandraConfig:[0m [37m+ cassandraHosts: [cassandra.onap][0m [37m+ cassandraPort: 9042[0m [37m+ localDataCenter: Pod[0m [37m+ reconnectTimeout : 30000[0m [37m+ socketReadTimeout: 20000[0m [37m+ socketConnectTimeout: 20000[0m [37m+ authenticate: true[0m [37m+ username: asdc_user[0m [37m+ password: Aa1234%^![0m [37m+ ssl: false[0m [37m+ truststorePath : /var/lib/jetty/etc/truststore[0m [37m+ truststorePassword : Aa123456[0m [37m+ keySpaces:[0m [37m+ - { name: dox, replicationStrategy: NetworkTopologyStrategy, replicationInfo: ['Pod','3']}[0m [37m+ - { name: sdcaudit, replicationStrategy: NetworkTopologyStrategy, replicationInfo: ['Pod','3']}[0m [37m+ - { name: sdcartifact, replicationStrategy: NetworkTopologyStrategy, replicationInfo: ['Pod','3']}[0m [37m+ - { name: sdccomponent, replicationStrategy: NetworkTopologyStrategy, replicationInfo: ['Pod','3']}[0m [37m+ - { name: sdcrepository, replicationStrategy: NetworkTopologyStrategy, replicationInfo: ['Pod','3']}[0m [37m+[0m [37m+[0m [37m+licenseTypes:[0m [37m+ - User[0m [37m+ - Installation[0m [37m+ - CPU[0m [37m+[0m [37m+#Deployment artifacts placeHolder[0m [37m+resourceTypes: &allResourceTypes[0m [37m+ - VFC[0m [37m+ - CP[0m [37m+ - VL[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - VFCMT[0m [37m+ - Abstract[0m [37m+ - CVFC[0m [37m+ - Configuration[0m [37m+ - ServiceProxy[0m [37m+ - PNF[0m [37m+[0m [37m+componentAllowedInstanceTypes:[0m [37m+ Resource:[0m [37m+ VF:[0m [37m+ - VFC[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - CP[0m [37m+ - PNF[0m [37m+ - CVFC[0m [37m+ - VL[0m [37m+ - Configuration[0m [37m+ - ServiceProxy[0m [37m+ - Abstract[0m [37m+ CVFC:[0m [37m+ - VFC[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - CP[0m [37m+ - PNF[0m [37m+ - CVFC[0m [37m+ - VL[0m [37m+ - ServiceProxy[0m [37m+ - Abstract[0m [37m+ PNF:[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - CP[0m [37m+ - PNF[0m [37m+ - CVFC[0m [37m+ - VL[0m [37m+ - Configuration[0m [37m+ - ServiceProxy[0m [37m+ - Abstract[0m [37m+ CR:[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - CP[0m [37m+ - PNF[0m [37m+ - CVFC[0m [37m+ - VL[0m [37m+ - Configuration[0m [37m+ - ServiceProxy[0m [37m+ - Abstract[0m [37m+ VL:[0m [37m+ - VL[0m [37m+ Service:[0m [37m+ "*":[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - CP[0m [37m+ - PNF[0m [37m+ - CVFC[0m [37m+ - VL[0m [37m+ - Configuration[0m [37m+ - ServiceProxy[0m [37m+ - Abstract[0m [37m+[0m [37m+artifacts:[0m [37m+ - type: CONTROLLER_BLUEPRINT_ARCHIVE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - zip[0m [37m+ - type: HELM[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - tgz[0m [37m+ - type: YANG_XML[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: VNF_CATALOG[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: MODEL_INVENTORY_PROFILE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: MODEL_QUERY_SPEC[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: UCPE_LAYER_2_CONFIGURATION[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ #AAI Artifacts[0m [37m+ - type: AAI_SERVICE_MODEL[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: AAI_VF_MODULE_MODEL[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: AAI_VF_INSTANCE_MODEL[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ #Plan[0m [37m+ - type: PLAN[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ - RESOURCE[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFC[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: WORKFLOW[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VFC[0m [37m+ - CP[0m [37m+ - VL[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - VFCMT[0m [37m+ - Abstract[0m [37m+ - CVFC[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - type: HEAT[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: HEAT_VOL[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: HEAT_NET[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: HEAT_NESTED[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: HEAT_ARTIFACT[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: CLOUD_TECHNOLOGY_SPECIFIC_ARTIFACT[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - zip[0m [37m+ - tgz[0m [37m+ - csar[0m [37m+ - type: VNF_CATALOG[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: VF_LICENSE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: VENDOR_LICENSE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: MODEL_INVENTORY_PROFILE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: MODEL_QUERY_SPEC[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: LIFECYCLE_OPERATIONS[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFC[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: VES_EVENTS[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ - VFC[0m [37m+ - CP[0m [37m+ - VL[0m [37m+ - VF[0m [37m+ - CR[0m [37m+ - VFCMT[0m [37m+ - Abstract[0m [37m+ - CVFC[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: PERFORMANCE_COUNTER[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - csv[0m [37m+ - type: APPC_CONFIG[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ acceptedTypes:[0m [37m+ - type: DCAE_TOSCA[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFCMT[0m [37m+ acceptedTypes:[0m [37m+ - yml[0m [37m+ - yaml[0m [37m+ - type: DCAE_JSON[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFCMT[0m [37m+ acceptedTypes:[0m [37m+ - json[0m [37m+ - type: DCAE_POLICY[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFCMT[0m [37m+ acceptedTypes:[0m [37m+ - emf[0m [37m+ - type: DCAE_DOC[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFCMT[0m [37m+ acceptedTypes:[0m [37m+ - type: DCAE_EVENT[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFCMT[0m [37m+ acceptedTypes:[0m [37m+ - type: AAI_VF_MODEL[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: AAI_VF_MODULE_MODEL[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ acceptedTypes:[0m [37m+ - xml[0m [37m+ - type: OTHER[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VFC[0m [37m+ - CVFC[0m [37m+ - CP[0m [37m+ - VL[0m [37m+ - VF[0m [37m+ - VFCMT[0m [37m+ - Abstract[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - type: SNMP_POLL[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: SNMP_TRAP[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: PM_DICTIONARY[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: YANG_MODULE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - yang[0m [37m+ - type: ANSIBLE_PLAYBOOK[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: ONBOARDED_PACKAGE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - csar[0m [37m+ - zip[0m [37m+ - type: ETSI_PACKAGE[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - csar[0m [37m+ - zip[0m [37m+ - type: HEAT_ENV[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - env[0m [37m+ - type: VF_MODULES_METADATA[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - json[0m [37m+ - type: DCAE_INVENTORY_TOSCA[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - yml[0m [37m+ - yaml[0m [37m+ - type: DCAE_INVENTORY_JSON[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - json[0m [37m+ - type: DCAE_INVENTORY_POLICY[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - emf[0m [37m+ - type: DCAE_INVENTORY_DOC[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - type: DCAE_INVENTORY_BLUEPRINT[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - type: DCAE_INVENTORY_EVENT[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - type: CHEF[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: PUPPET[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: SHELL[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: YANG[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: BPEL[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: DG_XML[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: MURANO_PKG[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: PNF_SW_INFORMATION[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: GUIDE[0m [37m+ categories:[0m [37m+ - INFORMATIONAL[0m [37m+ componentTypes:[0m [37m+ - RESOURCE[0m [37m+ resourceTypes:[0m [37m+ - VF[0m [37m+ - VFC[0m [37m+ - CVFC[0m [37m+ - PNF[0m [37m+ acceptedTypes:[0m [37m+ - yaml[0m [37m+ - yml[0m [37m+ - type: FLOW[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - type: SHELL_SCRIPT[0m [37m+ categories:[0m [37m+ - DEPLOYMENT[0m [37m+ componentTypes:[0m [37m+ - SERVICE[0m [37m+ - SERVICE_INSTANCE[0m [37m+ - RESOURCE[0m [37m+ - RESOURCE_INSTANCE[0m [37m+ resourceTypes: *allResourceTypes[0m [37m+ acceptedTypes:[0m [37m+ - sh[0m [37m+ - type: TOSCA_CSAR[0m [37m+ categories:[0m [37m+ - TOSCA[0m [37m+ componentTypes:[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - csar[0m [37m+ - type: TOSCA_TEMPLATE[0m [37m+ categories:[0m [37m+ - TOSCA[0m [37m+ componentTypes:[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - yml[0m [37m+ - yaml[0m [37m+ - type: NETWORK_CALL_FLOW[0m [37m+ categories:[0m [37m+ componentTypes:[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+ - type: ICON[0m [37m+ categories:[0m [37m+ componentTypes:[0m [37m+ resourceTypes:[0m [37m+ acceptedTypes:[0m [37m+[0m [37m+deploymentResourceArtifacts:[0m [37m+[0m [37m+deploymentResourceInstanceArtifacts:[0m [37m+ heatEnv:[0m [37m+ displayName: "HEAT ENV"[0m [37m+ type: HEAT_ENV[0m [37m+ description: "Auto-generated HEAT Environment deployment artifact"[0m [37m+ fileExtension: "env"[0m [37m+ VfHeatEnv:[0m [37m+ displayName: "VF HEAT ENV"[0m [37m+ type: HEAT_ENV[0m [37m+ description: "VF Auto-generated HEAT Environment deployment artifact"[0m [37m+ fileExtension: "env"[0m [37m+[0m [37m+#tosca artifacts placeholders[0m [37m+toscaArtifacts:[0m [37m+ assetToscaTemplate:[0m [37m+ artifactName: -template.yml[0m [37m+ displayName: Tosca Template[0m [37m+ type: TOSCA_TEMPLATE[0m [37m+ description: TOSCA representation of the asset[0m [37m+ assetToscaCsar:[0m [37m+ artifactName: -csar.csar[0m [37m+ displayName: Tosca Model[0m [37m+ type: TOSCA_CSAR[0m [37m+ description: TOSCA definition package of the asset[0m [37m+[0m [37m+#Informational artifacts placeHolder[0m [37m+excludeResourceCategory:[0m [37m+ - Generic[0m [37m+excludeResourceType:[0m [37m+ - PNF[0m [37m+ - CR[0m [37m+informationalResourceArtifacts:[0m [37m+ features:[0m [37m+ displayName: Features[0m [37m+ type: OTHER[0m [37m+ capacity:[0m [37m+ displayName: Capacity[0m [37m+ type: OTHER[0m [37m+ vendorTestResult:[0m [37m+ displayName: Vendor Test Result[0m [37m+ type: OTHER[0m [37m+ testScripts:[0m [37m+ displayName: Test Scripts[0m [37m+ type: OTHER[0m [37m+ CloudQuestionnaire:[0m [37m+ displayName: Cloud Questionnaire (completed)[0m [37m+ type: OTHER[0m [37m+ HEATTemplateFromVendor:[0m [37m+ displayName: HEAT Template from Vendor[0m [37m+ type: HEAT[0m [37m+ resourceSecurityTemplate:[0m [37m+ displayName: Resource Security Template[0m [37m+ type: OTHER[0m [37m+[0m [37m+excludeServiceCategory:[0m [37m+[0m [37m+informationalServiceArtifacts:[0m [37m+ serviceArtifactPlan:[0m [37m+ displayName: Service Artifact Plan[0m [37m+ type: OTHER[0m [37m+ summaryOfImpactsToECOMPElements:[0m [37m+ displayName: Summary of impacts to ECOMP elements,OSSs, BSSs[0m [37m+ type: OTHER[0m [37m+ controlLoopFunctions:[0m [37m+ displayName: Control Loop Functions[0m [37m+ type: OTHER[0m [37m+ dimensioningInfo:[0m [37m+ displayName: Dimensioning Info[0m [37m+ type: OTHER[0m [37m+ affinityRules:[0m [37m+ displayName: Affinity Rules[0m [37m+ type: OTHER[0m [37m+ operationalPolicies:[0m [37m+ displayName: Operational Policies[0m [37m+ type: OTHER[0m [37m+ serviceSpecificPolicies:[0m [37m+ displayName: Service-specific Policies[0m [37m+ type: OTHER[0m [37m+ engineeringRules:[0m [37m+ displayName: Engineering Rules (ERD)[0m [37m+ type: OTHER[0m [37m+ distributionInstructions:[0m [37m+ displayName: Distribution Instructions[0m [37m+ type: OTHER[0m [37m+ certificationTestResults:[0m [37m+ displayName: TD Certification Test Results[0m [37m+ type: OTHER[0m [37m+ deploymentVotingRecord:[0m [37m+ displayName: Deployment Voting Record[0m [37m+ type: OTHER[0m [37m+ serviceQuestionnaire:[0m [37m+ displayName: Service Questionnaire[0m [37m+ type: OTHER[0m [37m+ serviceSecurityTemplate:[0m [37m+ displayName: Service Security Template[0m [37m+ type: OTHER[0m [37m+[0m [37m+serviceApiArtifacts:[0m [37m+ configuration:[0m [37m+ displayName: Configuration[0m [37m+ type: OTHER[0m [37m+ instantiation:[0m [37m+ displayName: Instantiation[0m [37m+ type: OTHER[0m [37m+ monitoring:[0m [37m+ displayName: Monitoring[0m [37m+ type: OTHER[0m [37m+ reporting:[0m [37m+ displayName: Reporting[0m [37m+ type: OTHER[0m [37m+ logging:[0m [37m+ displayName: Logging[0m [37m+ type: OTHER[0m [37m+ testing:[0m [37m+ displayName: Testing[0m [37m+ type: OTHER[0m [37m+[0m [37m+[0m [37m+additionalInformationMaxNumberOfKeys: 50[0m [37m+[0m [37m+systemMonitoring:[0m [37m+ enabled: false[0m [37m+ isProxy: false[0m [37m+ probeIntervalInSeconds: 15[0m [37m+heatArtifactDeploymentTimeout:[0m [37m+ defaultMinutes: 30[0m [37m+ minMinutes: 1[0m [37m+ maxMinutes: 120[0m [37m+[0m [37m+unLoggedUrls:[0m [37m+ - /sdc2/rest/monitoring[0m [37m+ - /sdc2/rest/healthCheck[0m [37m+[0m [37m+cleanComponentsConfiguration:[0m [37m+ cleanIntervalInMinutes: 1440[0m [37m+ componentsToClean:[0m [37m+ - Resource[0m [37m+ - Service[0m [37m+[0m [37m+artifactsIndex: resources[0m [37m+[0m [37m+heatEnvArtifactHeader: ""[0m [37m+heatEnvArtifactFooter: ""[0m [37m+[0m [37m+onboarding:[0m [37m+ host: sdc-onboarding-be.onap[0m [37m+ protocol: https[0m [37m+ port: 8445[0m [37m+ downloadCsarUri: "/onboarding-api/v1.0/vendor-software-products/packages"[0m [37m+ healthCheckUri: "/onboarding-api/v1.0/healthcheck"[0m [37m+[0m [37m+# #GSS IDNS[0m [37m+switchoverDetector:[0m [37m+ gBeFqdn:[0m [37m+ gFeFqdn:[0m [37m+ beVip: 1.2.3.4[0m [37m+ feVip: 1.2.3.4[0m [37m+ beResolveAttempts: 3[0m [37m+ feResolveAttempts: 3[0m [37m+ enabled: false[0m [37m+ interval: 60[0m [37m+ changePriorityUser: ecompasdc[0m [37m+ changePriorityPassword: ecompasdc123[0m [37m+ publishNetworkUrl:[0m [37m+ publishNetworkBody: '{"note":"comment"}'[0m [37m+ groups:[0m [37m+ beSet: { changePriorityUrl: "", changePriorityBody: '{"name":"","uri":"","no_ad_redirection":false,"v4groups":{"failover_groups":["","","failover_policy":["FAILALL"]},"comment":"","intended_app_proto":"DNS"}'}[0m [37m+ feSet: { changePriorityUrl: "", changePriorityBody: '{"name":"","uri":"","no_ad_redirection":false,"v4groups":{"failover_groups":["",""],"failover_policy":["FAILALL"]},"comment":"","intended_app_proto":"DNS"}'}[0m [37m+[0m [37m+applicationL1Cache:[0m [37m+ datatypes:[0m [37m+ enabled: true[0m [37m+ firstRunDelay: 10[0m [37m+ pollIntervalInSec: 60[0m [37m+[0m [37m+applicationL2Cache:[0m [37m+ enabled: false[0m [37m+ catalogL1Cache:[0m [37m+ enabled: false[0m [37m+ resourcesSizeInCache: 300[0m [37m+ servicesSizeInCache: 200[0m [37m+ productsSizeInCache: 100[0m [37m+ queue:[0m [37m+ syncIntervalInSecondes: 43200[0m [37m+ waitOnShutDownInMinutes: 10[0m [37m+ numberOfCacheWorkers: 4[0m [37m+[0m [37m+toscaValidators:[0m [37m+ stringMaxLength: 2500[0m [37m+[0m [37m+disableAudit: false[0m [37m+[0m [37m+consumerBusinessLogic: true[0m [37m+[0m [37m+vfModuleProperties:[0m [37m+ min_vf_module_instances:[0m [37m+ forBaseModule: 1[0m [37m+ forNonBaseModule: 0[0m [37m+ max_vf_module_instances:[0m [37m+ forBaseModule: 1[0m [37m+ forNonBaseModule:[0m [37m+ initial_count:[0m [37m+ forBaseModule: 1[0m [37m+ forNonBaseModule: 0[0m [37m+ vf_module_type:[0m [37m+ forBaseModule: Base[0m [37m+ forNonBaseModule: Expansion[0m [37m+[0m [37m+genericAssetNodeTypes:[0m [37m+ VFC: org.openecomp.resource.abstract.nodes.VFC[0m [37m+ CVFC: org.openecomp.resource.abstract.nodes.VFC[0m [37m+ VF : org.openecomp.resource.abstract.nodes.VF[0m [37m+ CR : org.openecomp.resource.abstract.nodes.CR[0m [37m+ PNF: org.openecomp.resource.abstract.nodes.PNF[0m [37m+ Service: org.openecomp.resource.abstract.nodes.service[0m [37m+ ETSI NFV Network Service: tosca.nodes.nfv.NS[0m [37m+[0m [37m+serviceNodeTypes:[0m [37m+ ETSI NFV Network Service: tosca.nodes.nfv.NS[0m [37m+[0m [37m+workloadContext: Production[0m [37m+[0m [37m+environmentContext:[0m [37m+ defaultValue: General_Revenue-Bearing[0m [37m+ validValues:[0m [37m+ - Critical_Revenue-Bearing[0m [37m+ - Vital_Revenue-Bearing[0m [37m+ - Essential_Revenue-Bearing[0m [37m+ - Important_Revenue-Bearing[0m [37m+ - Needed_Revenue-Bearing[0m [37m+ - Useful_Revenue-Bearing[0m [37m+ - General_Revenue-Bearing[0m [37m+ - Critical_Non-Revenue[0m [37m+ - Vital_Non-Revenue[0m [37m+ - Essential_Non-Revenue[0m [37m+ - Important_Non-Revenue[0m [37m+ - Needed_Non-Revenue[0m [37m+ - Useful_Non-Revenue[0m [37m+ - General_Non-Revenue[0m [37m+[0m [37m+gabConfig:[0m [37m+ - artifactType: 'VES_EVENTS'[0m [37m+ pathsAndNamesDefinitions:[0m [37m+ -[0m [37m+ friendlyName: "Action"[0m [37m+ path: "event.action[2]"[0m [37m+ searchable: true[0m [37m+ -[0m [37m+ friendlyName: "Comment"[0m [37m+ path: "event.comment"[0m [37m+ searchable: true[0m [37m+ -[0m [37m+ friendlyName: "Alarm Additional Information"[0m [37m+ path: "event.structure.faultFields.structure.alarmAdditionalInformation.comment"[0m [37m+ searchable: true[0m [37m+ - artifactType: 'PM_DICTIONARY'[0m [37m+ pathsAndNamesDefinitions:[0m [37m+ -[0m [37m+ friendlyName: "measType"[0m [37m+ path: "pmMetaData.pmFields.measType"[0m [37m+ searchable: true[0m [37m+ -[0m [37m+ friendlyName: "measDescription"[0m [37m+ path: "pmMetaData.pmFields.measDescription"[0m [37m+ searchable: true[0m [37m+ -[0m [37m+ friendlyName: "measCondition"[0m [37m+ path: "pmMetaData.pmFields.measCondition"[0m [37m+ searchable: false[0m [37m+ -[0m [37m+ friendlyName: "measResultUnits"[0m [37m+ path: "pmMetaData.pmFields.measResultUnits"[0m [37m+ searchable: false[0m [37m+ -[0m [37m+ friendlyName: "measResultRange"[0m [37m+ path: "pmMetaData.pmFields.measResultRange"[0m [37m+ searchable: false[0m [37m+ -[0m [37m+ friendlyName: "measObjClass"[0m [37m+ path: "pmMetaData.pmFields.measObjClass"[0m [37m+ searchable: true[0m [37m+ -[0m [37m+ friendlyName: "measCollectionMethod"[0m [37m+ path: "pmMetaData.pmFields.measCollectionMethod"[0m [37m+ searchable: false[0m [37m+ -[0m [37m+ friendlyName: "measInfoId"[0m [37m+ path: "pmMetaData.pmFields.measInfoId"[0m [37m+ searchable: true[0m [37m+ -[0m [37m+ friendlyName: "iMeasInfoId"[0m [37m+ path: "pmMetaData.pmFields.iMeasInfoId"[0m [37m+ searchable: false[0m [37m+[0m [37m+dmaapConsumerConfiguration:[0m [37m+ active: false[0m [37m+ hosts: localhost:3905[0m [37m+ consumerGroup: sdc[0m [37m+ consumerId: mama[0m [37m+ timeoutMs: 15000[0m [37m+ limit: 1[0m [37m+ pollingInterval: 2[0m [37m+ topic: topic[0m [37m+ latitude: 32.109333[0m [37m+ longitude: 34.855499[0m [37m+ version: 1.0[0m [37m+ serviceName: localhost/events[0m [37m+ environment: TEST[0m [37m+ partner: BOT_R[0m [37m+ routeOffer: MR1[0m [37m+ protocol: https[0m [37m+ contenttype: application/json[0m [37m+ dme2TraceOn: true[0m [37m+ aftEnvironment: AFTUAT[0m [37m+ aftDme2ConnectionTimeoutMs: 15000[0m [37m+ aftDme2RoundtripTimeoutMs: 240000[0m [37m+ aftDme2ReadTimeoutMs: 50000[0m [37m+ dme2preferredRouterFilePath: DME2preferredRouter.txt[0m [37m+ timeLimitForNotificationHandleMs: 120000[0m [37m+ credential:[0m [37m+ username: user[0m [37m+ password:[0m [37m+ aftDme2SslEnable: true[0m [37m+ aftDme2ClientKeystore: /var/lib/jetty/etc/truststore[0m [37m+ aftDme2ClientKeystorePassword: ""[0m [37m+ aftDme2ClientSslCertAlias: certman[0m [37m+[0m [37m+dmaapProducerConfiguration:[0m [37m+ active: true[0m [37m+ hosts: olsd004.com:3905[0m [37m+ consumerGroup: sdc-AUTO-1622160351[0m [37m+ consumerId: sdc-AUTO1-1622160351[0m [37m+ timeoutMs: 15000[0m [37m+ limit: 1[0m [37m+ pollingInterval: 2[0m [37m+ topic: SDC-FACADE-NOTIF-v1 [0m [37m+ latitude: 32.109333[0m [37m+ longitude: 34.855499[0m [37m+ version: 1.0[0m [37m+ serviceName: dmaap.com:3905/events[0m [37m+ environment: TEST[0m [37m+ partner: BOT_R[0m [37m+ routeOffer: MR1[0m [37m+ protocol: [0m [37m+ contenttype: application/json[0m [37m+ dme2TraceOn: true[0m [37m+ aftEnvironment: [0m [37m+ aftDme2ConnectionTimeoutMs: 15000[0m [37m+ aftDme2RoundtripTimeoutMs: 240000[0m [37m+ aftDme2ReadTimeoutMs: 50000[0m [37m+ dme2preferredRouterFilePath: [0m [37m+ timeLimitForNotificationHandleMs: 120000[0m [37m+ credential:[0m [37m+ username: user1@sdc.com[0m [37m+ password: password==[0m [37m+ aftDme2SslEnable: true[0m [37m+ aftDme2ClientKeystore: /var/lib/jetty/etc/truststore[0m [37m+ aftDme2ClientKeystorePassword: dmaap_truststore_pwd[0m [37m+ aftDme2ClientSslCertAlias: certman[0m [37m+[0m [37m+[0m [37m+# ToDo: AF - had to remove due to configuration laod class failure[0m [37m+#dmeConfiguration:[0m [37m+# lookupUriFormat: "http://DME2RESOLVE/service=%s/version=1.0.0/envContext=%s/routeOffer=DEFAULT"[0m [37m+# dme2Search: DME2SEARCH[0m [37m+# dme2Resolve: DME2RESOLVE[0m [37m+[0m [37m+excludedPolicyTypesMapping:[0m [37m+ # VF:[0m [37m+ # - a.b.c[0m [37m+ # - c.d.e[0m [37m+ #CR:[0m [37m+ # - x.y.z[0m [37m+[0m [37m+excludedGroupTypesMapping:[0m [37m+ CR:[0m [37m+ - org.openecomp.groups.VfModule[0m [37m+ - org.openecomp.groups.heat.HeatStack[0m [37m+ - org.openecomp.groups.Group[0m [37m+ - tosca.groups.Root[0m [37m+ PNF:[0m [37m+ - org.openecomp.groups.VfModule[0m [37m+ - org.openecomp.groups.heat.HeatStack[0m [37m+ - org.openecomp.groups.Group[0m [37m+ - tosca.groups.Root[0m [37m+ VF:[0m [37m+ - org.openecomp.groups.VfModule[0m [37m+ - org.openecomp.groups.heat.HeatStack[0m [37m+ - org.openecomp.groups.Group[0m [37m+ - tosca.groups.Root[0m [37m+ Service:[0m [37m+ - org.openecomp.groups.VfModule[0m [37m+ - org.openecomp.groups.heat.HeatStack[0m [37m+ - org.openecomp.groups.Group[0m [37m+ - tosca.groups.Root[0m [37m+[0m [37m+healthStatusExclude:[0m [37m+ - DE[0m [37m+ - DMAAP[0m [37m+ - DMAAP_PRODUCER[0m [37m+ - ON_BOARDING[0m [37m+ - DCAE[0m [37m+ - PORTAL[0m [37m+ - External API[0m [37m+[0m [37m+#Auto Healing[0m [37m+enableAutoHealing: false[0m [37m+appVersion: [0m [37m+[0m [37m+artifactGeneratorConfig: Artifact-Generator.properties[0m [37m+resourcesForUpgrade:[0m [37m+ 8.0:[0m [37m+ - org.openecomp.resource.cp.extCP[0m [37m+ - tosca.nodes.network.Network[0m [37m+ - tosca.nodes.network.Port[0m [37m+ - org.openecomp.resource.cp.nodes.network.SubInterface[0m [37m+skipUpgradeFailedVfs: true[0m [37m+skipUpgradeVSPs: true[0m [37m+autoHealingOwner: jh0003[0m [37m+supportAllottedResourcesAndProxy: true[0m [37m+deleteLockTimeoutInSeconds: 60[0m [37m+maxDeleteComponents: 10[0m [37m+[0m [37m+# Cadi filter (access restriction) Parameters[0m [37m+aafNamespace: com.onap.sdc[0m [37m+aafAuthNeeded: false[0m [37m+[0m [37m+cadiFilterParams:[0m [37m+ AFT_LATITUDE: "32.780140"[0m [37m+ AFT_LONGITUDE: "-96.800451"[0m [37m+ hostname: sdc-be.onap[0m [37m+ aaf_id: user[0m [37m+ aaf_env: TEST[0m [37m+ aaf_url: [0m [37m+ csp_domain: PROD[0m [37m+ cadi_keyfile: /var/lib/jetty/etc/org.onap.sdc.p12[0m [37m+ aaf_password: enc:AafEncriptedPassword[0m [37m+ cadi_loglevel: DEBUG[0m [37m+ AFT_ENVIRONMENT: AFTUAT[0m [37m+ cadiX509Issuers: CN=intermediateCA_1, OU=OSAAF, O=ONAP, C=US[0m [37m+ cadi_truststore: /var/lib/jetty/etc/org.onap.sdc.trust.jks[0m [37m+ cadi_truststore_password: changeit[0m [37m+[0m [37m+# This configuration entry lists all node type names prefix that shall be allowed on SDC.[0m [37m+definedResourceNamespace:[0m [37m+ - org.openecomp.resource.[0m [37m+[0m [37m+# This configuration entry lists all Directives values that shall be allowed on SDC.[0m [37m+directives:[0m [37m+ - select[0m [37m+ - selectable[0m [37m+ - substitute[0m [37m+ - substitutable[0m[2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] mode changed to 644 [2021-05-28T00:05:51+00:00] INFO: template[catalog-be-config] mode changed to 644 [32m- change mode from '' to '0644'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * template[distribution-engine-configuration] action create_if_missing[2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] created file /var/lib/jetty/config/catalog-be/distribution-engine-configuration.yaml [2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] created file /var/lib/jetty/config/catalog-be/distribution-engine-configuration.yaml [32m- create new file /var/lib/jetty/config/catalog-be/distribution-engine-configuration.yaml[0m[2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] updated file contents /var/lib/jetty/config/catalog-be/distribution-engine-configuration.yaml [2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] updated file contents /var/lib/jetty/config/catalog-be/distribution-engine-configuration.yaml [32m- update content in file /var/lib/jetty/config/catalog-be/distribution-engine-configuration.yaml from none to 44c609[0m [37m--- /var/lib/jetty/config/catalog-be/distribution-engine-configuration.yaml 2021-05-28 00:05:51.256667681 +0000[0m [37m+++ /var/lib/jetty/config/catalog-be/.chef-distribution-engine-configuration20210528-22-7gzez.yaml 2021-05-28 00:05:51.256667681 +0000[0m [37m@@ -1,81 +1,162 @@[0m [37m+uebServers:[0m [37m+ - message-router.onap[0m [37m+ - message-router.onap[0m [37m+[0m [37m+uebPublicKey: iPIxkpAMI8qTcQj8[0m [37m+uebSecretKey: Ehq3WyT4bkif4zwgEbvshGal[0m [37m+[0m [37m+distributionNotifTopicName: SDC-DISTR-NOTIF-TOPIC[0m [37m+distributionStatusTopicName: SDC-DISTR-STATUS-TOPIC[0m [37m+[0m [37m+initRetryIntervalSec: 5[0m [37m+initMaxIntervalSec: 60[0m [37m+[0m [37m+distribNotifServiceArtifactTypes:[0m [37m+ info:[0m [37m+ - MURANO-PKG[0m [37m+[0m [37m+distribNotifResourceArtifactTypes:[0m [37m+ lifecycle:[0m [37m+ - HEAT[0m [37m+ - DG-XML[0m [37m+[0m [37m+environments:[0m [37m+ - AUTO[0m [37m+[0m [37m+distributionStatusTopic:[0m [37m+ pollingIntervalSec: 60[0m [37m+ fetchTimeSec: 15[0m [37m+ consumerGroup: sdc-AUTO[0m [37m+ consumerId: sdc-AUTO1[0m [37m+[0m [37m+[0m [37m+distributionNotificationTopic:[0m [37m+ minThreadPoolSize: 0[0m [37m+ maxThreadPoolSize: 10[0m [37m+ maxWaitingAfterSendingSeconds: 5[0m [37m+[0m [37m+createTopic:[0m [37m+ partitionCount: 1[0m [37m+ replicationCount: 1[0m [37m+[0m [37m+startDistributionEngine: true[0m [37m+[0m [37m+#This is false by default, since ONAP Dmaap currently doesn't support https[0m [37m+useHttpsWithDmaap: false[0m [37m+opEnvRecoveryIntervalSec: 180[0m [37m+allowedTimeBeforeStaleSec: 300[0m [37m+[0m [37m+aaiConfig:[0m [37m+ httpRequestConfig:[0m [37m+ serverRootUrl: https://localhost:8443[0m [37m+ resourceNamespaces:[0m [37m+ operationalEnvironments: /aai/v12/cloud-infrastructure/operational-environments[0m [37m+[0m [37m+ httpClientConfig:[0m [37m+ timeouts:[0m [37m+ readTimeoutMs: 5000[0m [37m+ connectTimeoutMs: 1000[0m [37m+ clientCertificate:[0m [37m+ keyStore: /var/lib/jetty/base/be/etc/non-prod.jks[0m [37m+ keyStorePassword: hmXYcznAljMSisdy8zgcag==[0m [37m+ headers:[0m [37m+ X-FromAppId: asdc[0m [37m+ numOfRetries: 3[0m [37m+[0m [37m+msoConfig:[0m [37m+ httpRequestConfig:[0m [37m+ serverRootUrl: http://127.0.0.1:8080/onap/mso/infra/modelDistributions/v1[0m [37m+ resourceNamespaces:[0m [37m+ distributions: /distributions[0m [37m+[0m [37m+ httpClientConfig:[0m [37m+ timeouts:[0m [37m+ readTimeoutMs: 2000[0m [37m+ connectTimeoutMs: 500[0m [37m+ basicAuthorization:[0m [37m+ userName: asdc[0m [37m+ password: OTLEp5lfVhYdyw5EAtTUBQ==[0m [37m+ numOfRetries: 3[0m [37m+ [0m [37m+currentArtifactInstallationTimeout: 120[0m[2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] mode changed to 644 [2021-05-28T00:05:51+00:00] INFO: template[distribution-engine-configuration] mode changed to 644 [32m- change mode from '' to '0644'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0mRecipe: sdc-catalog-be::BE_3_locate_keystore[0m * directory[Jetty_etcdir_creation] action create[2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] created directory /var/lib/jetty/etc [2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] created directory /var/lib/jetty/etc [32m- create new directory /var/lib/jetty/etc[0m[2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] mode changed to 755 [2021-05-28T00:05:51+00:00] INFO: directory[Jetty_etcdir_creation] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] action create[2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] created file /var/lib/jetty/etc/org.onap.sdc.p12 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] created file /var/lib/jetty/etc/org.onap.sdc.p12 [32m- create new file /var/lib/jetty/etc/org.onap.sdc.p12[0m[2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] updated file contents /var/lib/jetty/etc/org.onap.sdc.p12 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] updated file contents /var/lib/jetty/etc/org.onap.sdc.p12 [32m- update content in file /var/lib/jetty/etc/org.onap.sdc.p12 from none to 8514e0[0m [37m(new content is binary, diff output suppressed)[0m[2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] mode changed to 755 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.p12] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] action create[2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] created file /var/lib/jetty/etc/org.onap.sdc.trust.jks [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] created file /var/lib/jetty/etc/org.onap.sdc.trust.jks [32m- create new file /var/lib/jetty/etc/org.onap.sdc.trust.jks[0m[2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] updated file contents /var/lib/jetty/etc/org.onap.sdc.trust.jks [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] updated file contents /var/lib/jetty/etc/org.onap.sdc.trust.jks [32m- update content in file /var/lib/jetty/etc/org.onap.sdc.trust.jks from none to e36d22[0m [37m(new content is binary, diff output suppressed)[0m[2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] owner changed to 352070 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] group changed to 35953 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] mode changed to 755 [2021-05-28T00:05:51+00:00] INFO: cookbook_file[/var/lib/jetty/etc/org.onap.sdc.trust.jks] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0mRecipe: sdc-catalog-be::BE_4_jetty_Modules[0m * bash[create-jetty-modules] action run [execute] INFO : deploy already enabled by [${jetty.base}/start.d/deploy.ini] INFO : Base directory was not modified INFO : http already enabled by [${jetty.base}/start.d/http.ini] INFO : console-capture initialized in ${jetty.base}/start.d/console-capture.ini INFO : https initialized in ${jetty.base}/start.d/https.ini INFO : ssl transitively enabled, ini template available with --add-to-start=ssl INFO : setuid initialized in ${jetty.base}/start.d/setuid.ini MKDIR : ${jetty.base}/logs COPY : ${jetty.home}/modules/ssl/keystore to ${jetty.base}/etc/keystore INFO : Base directory was modified [2021-05-28T00:05:52+00:00] INFO: bash[create-jetty-modules] ran successfully [2021-05-28T00:05:52+00:00] INFO: bash[create-jetty-modules] ran successfully [32m- execute "bash" "/tmp/chef-script20210528-22-1pdxpn9"[0m [0m * template[http-ini] action create[2021-05-28T00:05:52+00:00] INFO: template[http-ini] backed up to /var/lib/jetty/chef-solo/backup/var/lib/jetty/start.d/http.ini.chef-20210528000552.472201 [2021-05-28T00:05:52+00:00] INFO: template[http-ini] backed up to /var/lib/jetty/chef-solo/backup/var/lib/jetty/start.d/http.ini.chef-20210528000552.472201 [2021-05-28T00:05:52+00:00] INFO: template[http-ini] updated file contents /var/lib/jetty/start.d/http.ini [2021-05-28T00:05:52+00:00] INFO: template[http-ini] updated file contents /var/lib/jetty/start.d/http.ini [32m- update content in file /var/lib/jetty/start.d/http.ini from 9c50e4 to 600ef1[0m [37m--- /var/lib/jetty/start.d/http.ini 2020-09-11 06:28:08.000000000 +0000[0m [37m+++ /var/lib/jetty/start.d/.chef-http20210528-22-2fe668.ini 2021-05-28 00:05:52.464746634 +0000[0m [37m@@ -1,10 +1,6 @@[0m [37m-# --------------------------------------- [0m [37m+# ---------------------------------------[0m [37m # Module: http[0m [37m-# Enables an HTTP connector on the server.[0m [37m-# By default HTTP/1 is support, but HTTP2C can[0m [37m-# be added to the connector with the http2c module.[0m [37m-# --------------------------------------- [0m [37m---module=http[0m [37m+#--module=http[0m [37m [0m [37m ### HTTP Connector Configuration[0m [37m [0m [37m@@ -12,11 +8,14 @@[0m [37m # jetty.http.host=0.0.0.0[0m [37m [0m [37m ## Connector port to listen on[0m [37m-# jetty.http.port=8080[0m [37m+jetty.http.port=8080[0m [37m [0m [37m ## Connector idle timeout in milliseconds[0m [37m-# jetty.http.idleTimeout=30000[0m [37m+jetty.http.idleTimeout=30000[0m [37m [0m [37m+## Connector socket linger time in seconds (-1 to disable)[0m [37m+# jetty.http.soLingerTime=-1[0m [37m+[0m [37m ## Number of acceptors (-1 picks default based on number of cores)[0m [37m # jetty.http.acceptors=-1[0m [37m [0m [37m@@ -24,30 +23,8 @@[0m [37m # jetty.http.selectors=-1[0m [37m [0m [37m ## ServerSocketChannel backlog (0 picks platform default)[0m [37m-# jetty.http.acceptQueueSize=0[0m [37m+# jetty.http.acceptorQueueSize=0[0m [37m [0m [37m ## Thread priority delta to give to acceptor threads[0m [37m # jetty.http.acceptorPriorityDelta=0[0m [37m-[0m [37m-## The requested maximum length of the queue of incoming connections.[0m [37m-# jetty.http.acceptQueueSize=0[0m [37m-[0m [37m-## Enable/disable the SO_REUSEADDR socket option.[0m [37m-# jetty.http.reuseAddress=true[0m [37m-[0m [37m-## Enable/disable TCP_NODELAY on accepted sockets.[0m [37m-# jetty.http.acceptedTcpNoDelay=true[0m [37m-[0m [37m-## The SO_RCVBUF option to set on accepted sockets. A value of -1 indicates that it is left to its default value.[0m [37m-# jetty.http.acceptedReceiveBufferSize=-1[0m [37m-[0m [37m-## The SO_SNDBUF option to set on accepted sockets. A value of -1 indicates that it is left to its default value.[0m [37m-# jetty.http.acceptedSendBufferSize=-1[0m [37m-[0m [37m-## Connect Timeout in milliseconds[0m [37m-# jetty.http.connectTimeout=15000[0m [37m-[0m [37m-## HTTP Compliance: RFC7230, RFC7230_LEGACY, RFC2616, RFC2616_LEGACY, LEGACY or CUSTOMn[0m [37m-# jetty.http.compliance=RFC7230_LEGACY[0m [37m-[0m[2021-05-28T00:05:52+00:00] INFO: template[http-ini] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: template[http-ini] mode changed to 755 [32m- change mode from '0644' to '0755'[0m [0m * template[https-ini] action create[2021-05-28T00:05:52+00:00] INFO: template[https-ini] backed up to /var/lib/jetty/chef-solo/backup/var/lib/jetty/start.d/https.ini.chef-20210528000552.484732 [2021-05-28T00:05:52+00:00] INFO: template[https-ini] backed up to /var/lib/jetty/chef-solo/backup/var/lib/jetty/start.d/https.ini.chef-20210528000552.484732 [2021-05-28T00:05:52+00:00] INFO: template[https-ini] updated file contents /var/lib/jetty/start.d/https.ini [2021-05-28T00:05:52+00:00] INFO: template[https-ini] updated file contents /var/lib/jetty/start.d/https.ini [32m- update content in file /var/lib/jetty/start.d/https.ini from 1167e5 to e03583[0m [37m--- /var/lib/jetty/start.d/https.ini 2021-05-28 00:05:52.420743758 +0000[0m [37m+++ /var/lib/jetty/start.d/.chef-https20210528-22-16gjrwt.ini 2021-05-28 00:05:52.480747680 +0000[0m [37m@@ -1,8 +1,30 @@[0m [37m-# --------------------------------------- [0m [37m+# ---------------------------------------[0m [37m # Module: https[0m [37m-# Adds HTTPS protocol support to the TLS(SSL) Connector[0m [37m-# --------------------------------------- [0m [37m --module=https[0m [37m [0m [37m+### HTTPS Connector Configuration[0m [37m [0m [37m+## Connector host/address to bind to[0m [37m+# jetty.https.host=0.0.0.0[0m [37m+[0m [37m+## Connector port to listen on[0m [37m+jetty.https.port=8443[0m [37m+[0m [37m+## Connector idle timeout in milliseconds[0m [37m+jetty.https.idleTimeout=30000[0m [37m+[0m [37m+## Connector socket linger time in seconds (-1 to disable)[0m [37m+# jetty.https.soLingerTime=-1[0m [37m+[0m [37m+## Number of acceptors (-1 picks default based on number of cores)[0m [37m+# jetty.https.acceptors=-1[0m [37m+[0m [37m+## Number of selectors (-1 picks default based on number of cores)[0m [37m+# jetty.https.selectors=-1[0m [37m+[0m [37m+## ServerSocketChannel backlog (0 picks platform default)[0m [37m+# jetty.https.acceptorQueueSize=0[0m [37m+[0m [37m+## Thread priority delta to give to acceptor threads[0m [37m+# jetty.https.acceptorPriorityDelta=0[0m[2021-05-28T00:05:52+00:00] INFO: template[https-ini] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: template[https-ini] mode changed to 755 [32m- change mode from '0644' to '0755'[0m [0m * template[ssl-ini] action create[2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] created file /var/lib/jetty/start.d/ssl.ini [2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] created file /var/lib/jetty/start.d/ssl.ini [32m- create new file /var/lib/jetty/start.d/ssl.ini[0m[2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] updated file contents /var/lib/jetty/start.d/ssl.ini [2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] updated file contents /var/lib/jetty/start.d/ssl.ini [32m- update content in file /var/lib/jetty/start.d/ssl.ini from none to 88644a[0m [37m--- /var/lib/jetty/start.d/ssl.ini 2021-05-28 00:05:52.492748464 +0000[0m [37m+++ /var/lib/jetty/start.d/.chef-ssl20210528-22-z4625w.ini 2021-05-28 00:05:52.492748464 +0000[0m [37m@@ -1,90 +1,180 @@[0m [37m+# ---------------------------------------[0m [37m+# Module: ssl[0m [37m+--module=ssl[0m [37m+[0m [37m+### TLS(SSL) Connector Configuration[0m [37m+[0m [37m+## Connector host/address to bind to[0m [37m+# jetty.ssl.host=0.0.0.0[0m [37m+[0m [37m+## Connector port to listen on[0m [37m+jetty.ssl.port=8443[0m [37m+[0m [37m+## Connector idle timeout in milliseconds[0m [37m+# jetty.ssl.idleTimeout=30000[0m [37m+[0m [37m+## Connector socket linger time in seconds (-1 to disable)[0m [37m+# jetty.ssl.soLingerTime=-1[0m [37m+[0m [37m+## Number of acceptors (-1 picks default based on number of cores)[0m [37m+# jetty.ssl.acceptors=-1[0m [37m+[0m [37m+## Number of selectors (-1 picks default based on number of cores)[0m [37m+# jetty.ssl.selectors=-1[0m [37m+[0m [37m+## ServerSocketChannel backlog (0 picks platform default)[0m [37m+# jetty.ssl.acceptorQueueSize=0[0m [37m+[0m [37m+## Thread priority delta to give to acceptor threads[0m [37m+# jetty.ssl.acceptorPriorityDelta=0[0m [37m+[0m [37m+## Whether request host names are checked to match any SNI names[0m [37m+# jetty.ssl.sniHostCheck=true[0m [37m+[0m [37m+## max age in seconds for a Strict-Transport-Security response header (default -1)[0m [37m+# jetty.ssl.stsMaxAgeSeconds=31536000[0m [37m+[0m [37m+## include subdomain property in any Strict-Transport-Security header (default false)[0m [37m+# jetty.ssl.stsIncludeSubdomains=true[0m [37m+[0m [37m+### SslContextFactory Configuration[0m [37m+## Note that OBF passwords are not secure, just protected from casual observation[0m [37m+## See http://www.eclipse.org/jetty/documentation/current/configuring-security-secure-passwords.html[0m [37m+[0m [37m+## Keystore file path (relative to $jetty.base)[0m [37m+jetty.sslContext.keyStorePath=etc/org.onap.sdc.p12[0m [37m+[0m [37m+## Truststore file path (relative to $jetty.base)[0m [37m+jetty.sslContext.trustStorePath=etc/org.onap.sdc.trust.jks[0m [37m+[0m [37m+## Keystore password[0m [37m+# jetty.sslContext.keyStorePassword=OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4[0m [37m+jetty.sslContext.keyStorePassword=Mo$h9F*9ftBd?GbNxhQ:RefX[0m [37m+[0m [37m+## Keystore type and provider[0m [37m+# jetty.sslContext.keyStoreType=JKS[0m [37m+# jetty.sslContext.keyStoreProvider=[0m [37m+[0m [37m+## KeyManager password[0m [37m+# jetty.sslContext.keyManagerPassword=OBF:1u2u1wml1z7s1z7a1wnl1u2g[0m [37m+jetty.sslContext.keyManagerPassword=Mo$h9F*9ftBd?GbNxhQ:RefX[0m [37m+[0m [37m+## Truststore password[0m [37m+# jetty.sslContext.trustStorePassword=OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4[0m [37m+jetty.sslContext.trustStorePassword=m9{e,(X5fnDzsJ&Ll)sW3ieQ[0m [37m+[0m [37m+## Truststore type and provider[0m [37m+# jetty.sslContext.trustStoreType=JKS[0m [37m+# jetty.sslContext.trustStoreProvider=[0m [37m+[0m [37m+## whether client certificate authentication is required[0m [37m+# jetty.sslContext.needClientAuth=false[0m [37m+[0m [37m+## Whether client certificate authentication is desired[0m [37m+# jetty.sslContext.wantClientAuth=false[0m [37m+[0m [37m+## Whether cipher order is significant (since java 8 only)[0m [37m+# jetty.sslContext.useCipherSuitesOrder=true[0m [37m+[0m [37m+## To configure Includes / Excludes for Cipher Suites or Protocols see tweak-ssl.xml example at[0m [37m+## https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html#configuring-sslcontextfactory-cipherSuites[0m [37m+[0m [37m+## Set the size of the SslSession cache[0m [37m+# jetty.sslContext.sslSessionCacheSize=-1[0m [37m+[0m [37m+## Set the timeout (in seconds) of the SslSession cache timeout[0m [37m+# jetty.sslContext.sslSessionTimeout=-1[0m [37m+[0m [37m+## Allow SSL renegotiation[0m [37m+# jetty.sslContext.renegotiationAllowed=true[0m [37m+# jetty.sslContext.renegotiationLimit=5[0m[2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: template[ssl-ini] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0mRecipe: sdc-catalog-be::BE_6_setup_portal_and_key_properties[0m * template[template portal.properties] action create[2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] created file /var/lib/jetty/resources/portal.properties [2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] created file /var/lib/jetty/resources/portal.properties [32m- create new file /var/lib/jetty/resources/portal.properties[0m[2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] updated file contents /var/lib/jetty/resources/portal.properties [2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] updated file contents /var/lib/jetty/resources/portal.properties [32m- update content in file /var/lib/jetty/resources/portal.properties from none to 62974b[0m [37m--- /var/lib/jetty/resources/portal.properties 2021-05-28 00:05:52.500748987 +0000[0m [37m+++ /var/lib/jetty/resources/.chef-portal20210528-22-1gpq98c.properties 2021-05-28 00:05:52.500748987 +0000[0m [37m@@ -1,89 +1,178 @@[0m [37m+###[0m [37m+# ============LICENSE_START==========================================[0m [37m+# ONAP Portal SDK[0m [37m+# ===================================================================[0m [37m+# Copyright © 2017 AT&T Intellectual Property. All rights reserved.[0m [37m+# ===================================================================[0m [37m+#[0m [37m+# Unless otherwise specified, all software contained herein is licensed[0m [37m+# under the Apache License, Version 2.0 (the “License”);[0m [37m+# you may not use this software except in compliance with the License.[0m [37m+# You may obtain a copy of the License at[0m [37m+#[0m [37m+# http://www.apache.org/licenses/LICENSE-2.0[0m [37m+#[0m [37m+# Unless required by applicable law or agreed to in writing, software[0m [37m+# distributed under the License is distributed on an "AS IS" BASIS,[0m [37m+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.[0m [37m+# See the License for the specific language governing permissions and[0m [37m+# limitations under the License.[0m [37m+#[0m [37m+# Unless otherwise specified, all documentation contained herein is licensed[0m [37m+# under the Creative Commons License, Attribution 4.0 Intl. (the “License”);[0m [37m+# you may not use this documentation except in compliance with the License.[0m [37m+# You may obtain a copy of the License at[0m [37m+#[0m [37m+# https://creativecommons.org/licenses/by/4.0/[0m [37m+#[0m [37m+# Unless required by applicable law or agreed to in writing, documentation[0m [37m+# distributed under the License is distributed on an "AS IS" BASIS,[0m [37m+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.[0m [37m+# See the License for the specific language governing permissions and[0m [37m+# limitations under the License.[0m [37m+#[0m [37m+# ============LICENSE_END============================================[0m [37m+#[0m [37m+#[0m [37m+###[0m [37m+# Properties read by ECOMP Framework library, ecompFW.jar[0m [37m+[0m [37m+##########################################################################[0m [37m+# The following properties should NOT be changed by partner applications.[0m [37m+##########################################################################[0m [37m+[0m [37m+portal.api.prefix = /api[0m [37m+max.idle.time = 5[0m [37m+user.attribute.name = user_attribute[0m [37m+[0m [37m+#Use REST API instead of UEB to fetch the functional menu data[0m [37m+use_rest_for_functional_menu=true[0m [37m+[0m [37m+[0m [37m+##########################################################################[0m [37m+# The following properties MUST be changed by partner applications.[0m [37m+##########################################################################[0m [37m+[0m [37m+# Name of java class that implements the OnBoardingApiService interface.[0m [37m+portal.api.impl.class = org.openecomp.sdc.be.ecomp.PortalRestApiCentralServiceImpl[0m [37m+role_access_centralized = remote[0m [37m+[0m [37m+# Cookie set by CSP-SSO[0m [37m+#csp_cookie_name = onapCsp[0m [37m+[0m [37m+# CSP setting, most use PROD; DEV also recognized[0m [37m+csp_gate_keeper_prod_key = PROD[0m [37m+[0m [37m+# URL of the Portal where this app is onboarded[0m [37m+ecomp_redirect_url = https://portal-app.onap:30225/ONAPPORTAL/login.htm[0m [37m+[0m [37m+# URL of the ECOMP Portal REST API[0m [37m+ecomp_rest_url = https://portal-app.onap:8443/ONAPPORTAL/auxapi[0m [37m+[0m [37m+# Connection and Read timeout values[0m [37m+ext_req_connection_timeout = 15000[0m [37m+ext_req_read_timeout = 20000[0m [37m+[0m [37m+# Name of java class that implements the OnBoardingApiService interface.[0m [37m+portal.api.impl.class = org.openecomp.sdc.be.ecomp.PortalRestApiCentralServiceImpl[0m [37m+[0m [37m+#Portal user & key[0m [37m+portal_app_name = Ipwxi2oLvDxctMA1royaRw1W0jhucLx+grHzci3ePIA=[0m [37m+portal_pass = j85yNhyIs7zKYbR1VlwEfNhS6b7Om4l0Gx5O8931sCI=[0m [37m+portal_user = Ipwxi2oLvDxctMA1royaRw1W0jhucLx+grHzci3ePIA=[0m [37m+[0m [37m+[0m [37m+# UEB key generated while on-boarding[0m [37m+ueb_app_key = ""[0m [37m+[0m [37m+# Applications do not need to run a UEB listener after 1607.[0m [37m+ueb_listeners_enable = false[0m[2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: template[template portal.properties] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * template[template key.properties] action create[2021-05-28T00:05:52+00:00] INFO: template[template key.properties] created file /var/lib/jetty/resources/key.properties [2021-05-28T00:05:52+00:00] INFO: template[template key.properties] created file /var/lib/jetty/resources/key.properties [32m- create new file /var/lib/jetty/resources/key.properties[0m[2021-05-28T00:05:52+00:00] INFO: template[template key.properties] updated file contents /var/lib/jetty/resources/key.properties [2021-05-28T00:05:52+00:00] INFO: template[template key.properties] updated file contents /var/lib/jetty/resources/key.properties [32m- update content in file /var/lib/jetty/resources/key.properties from none to f2c564[0m [37m--- /var/lib/jetty/resources/key.properties 2021-05-28 00:05:52.512749772 +0000[0m [37m+++ /var/lib/jetty/resources/.chef-key20210528-22-1e9wwsu.properties 2021-05-28 00:05:52.512749772 +0000[0m [37m@@ -1,41 +1,82 @@[0m [37m+###[0m [37m+# ============LICENSE_START==========================================[0m [37m+# ONAP Portal SDK[0m [37m+# ===================================================================[0m [37m+# Copyright © 2017 AT&T Intellectual Property. All rights reserved.[0m [37m+# ===================================================================[0m [37m+#[0m [37m+# Unless otherwise specified, all software contained herein is licensed[0m [37m+# under the Apache License, Version 2.0 (the “License”);[0m [37m+# you may not use this software except in compliance with the License.[0m [37m+# You may obtain a copy of the License at[0m [37m+#[0m [37m+# http://www.apache.org/licenses/LICENSE-2.0[0m [37m+#[0m [37m+# Unless required by applicable law or agreed to in writing, software[0m [37m+# distributed under the License is distributed on an "AS IS" BASIS,[0m [37m+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.[0m [37m+# See the License for the specific language governing permissions and[0m [37m+# limitations under the License.[0m [37m+#[0m [37m+# Unless otherwise specified, all documentation contained herein is licensed[0m [37m+# under the Creative Commons License, Attribution 4.0 Intl. (the “License”);[0m [37m+# you may not use this documentation except in compliance with the License.[0m [37m+# You may obtain a copy of the License at[0m [37m+#[0m [37m+# https://creativecommons.org/licenses/by/4.0/[0m [37m+#[0m [37m+# Unless required by applicable law or agreed to in writing, documentation[0m [37m+# distributed under the License is distributed on an "AS IS" BASIS,[0m [37m+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.[0m [37m+# See the License for the specific language governing permissions and[0m [37m+# limitations under the License.[0m [37m+#[0m [37m+# ============LICENSE_END============================================[0m [37m+#[0m [37m+# [0m [37m+###[0m [37m+[0m [37m+# Properties read by the ECOMP Framework library (epsdk-fw)[0m [37m+[0m [37m+cipher.enc.key = AGLDdG4D04BKm2IxIWEr8o==[0m[2021-05-28T00:05:52+00:00] INFO: template[template key.properties] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[template key.properties] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[template key.properties] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[template key.properties] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[template key.properties] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: template[template key.properties] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0mRecipe: sdc-catalog-be::BE_7_logback[0m * cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] action create_if_missing[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] created file /var/lib/jetty/config/catalog-be/logback.xml [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] created file /var/lib/jetty/config/catalog-be/logback.xml [32m- create new file /var/lib/jetty/config/catalog-be/logback.xml[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] updated file contents /var/lib/jetty/config/catalog-be/logback.xml [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] updated file contents /var/lib/jetty/config/catalog-be/logback.xml [32m- update content in file /var/lib/jetty/config/catalog-be/logback.xml from none to 03b6e7[0m [37m--- /var/lib/jetty/config/catalog-be/logback.xml 2021-05-28 00:05:52.516750033 +0000[0m [37m+++ /var/lib/jetty/config/catalog-be/.chef-logback20210528-22-1kdbwdf.xml 2021-05-28 00:05:52.516750033 +0000[0m [37m@@ -1,253 +1,506 @@[0m [37m+[0m [37m+
[0m [37m+[0m [37m+ [0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] mode changed to 644 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/logback.xml] mode changed to 644 [32m- change mode from '' to '0644'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0mRecipe: sdc-catalog-be::BE_8_errors_config[0m * cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] action create_if_missing[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] created file /var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] created file /var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml [32m- create new file /var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] updated file contents /var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] updated file contents /var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml [32m- update content in file /var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml from none to fc2c33[0m [37m--- /var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml 2021-05-28 00:05:52.532751079 +0000[0m [37m+++ /var/lib/jetty/config/catalog-be/.chef-ecomp-error-configuration20210528-22-2tqfht.yaml 2021-05-28 00:05:52.532751079 +0000[0m [37m@@ -1,383 +1,766 @@[0m [37m+###########################################[0m [37m+# Note the conventions of the field values:[0m [37m+# type can be one of: CONFIG_ERROR, SYSTEM_ERROR, DATA_ERROR, CONNECTION_PROBLEM, AUTHENTICATION_PROBLEM[0m [37m+# severity can be one of: WARN, ERROR, FATAL[0m [37m+# alarmSeverity can be one of: CRITICAL,MAJOR,MINOR,INFORMATIONAL,NONE[0m [37m+# code is a unique integer in range of 3003-9999 (3000-3002 are occupied for internal usage)[0m [37m+# The above enumeration values are out-of-the-box and can be changed in code. [0m [37m+# In case of config and code mismatch, the appropriate error will be printed to log[0m [37m+#[0m [37m+## Range of BE codes - 3010-7999[0m [37m+[0m [37m+errors:[0m [37m+[0m [37m+ BeRestApiGeneralError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_4000,[0m [37m+ severity: ERROR,[0m [37m+ description: "Unexpected error during BE REST API execution",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+[0m [37m+ BeHealthCheckError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3010,[0m [37m+ severity: ERROR,[0m [37m+ description: "Error during BE Health Check",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+ [0m [37m+ BeInitializationError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_4019,[0m [37m+ severity: ERROR,[0m [37m+ description: "Catalog-BE was not initialized properly",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+[0m [37m+ BeResourceMissingError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3011,[0m [37m+ severity: ERROR,[0m [37m+ description: "Mandatory resource %s cannot be found in repository",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeServiceMissingError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3012,[0m [37m+ severity: ERROR,[0m [37m+ description: "Mandatory service %s cannot be found in repository",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedAddingResourceInstanceError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3013,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to add resource instance of resource %s to service %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeIncorrectServiceError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3014,[0m [37m+ severity: ERROR,[0m [37m+ description: "Service %s is not valid",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeRepositoryDeleteError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3015,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to delete object %s from repository",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+ [0m [37m+ BeRepositoryQueryError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3016,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to fetch from repository %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeInvalidConfigurationError: {[0m [37m+ type: CONFIG_ERROR,[0m [37m+ code: ASDC_3017,[0m [37m+ severity: FATAL,[0m [37m+ description: "Configuration parameter %s is invalid. Value configured is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeUebConnectionError: {[0m [37m+ type: CONNECTION_PROBLEM,[0m [37m+ code: ASDC_4001,[0m [37m+ severity: ERROR,[0m [37m+ description: "Connection problem towards U-EB server. Reason: %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeUebSystemError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3019,[0m [37m+ severity: ERROR,[0m [37m+ description: "Error occured during access to U-EB Server. Operation: %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeUebObjectNotFoundError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4005,[0m [37m+ severity: ERROR,[0m [37m+ description: "Error occured during access to U-EB Server. Data not found: %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+[0m [37m+ BeDistributionEngineSystemError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3021,[0m [37m+ severity: ERROR,[0m [37m+ description: "Error occured in Distribution Engine. Failed operation: %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeUebAuthenticationError: {[0m [37m+ type: AUTHENTICATION_PROBLEM,[0m [37m+ code: ASDC_4003,[0m [37m+ severity: ERROR,[0m [37m+ description: "Authentication problem towards U-EB server. Reason: %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeUebUnkownHostError: {[0m [37m+ type: CONNECTION_PROBLEM,[0m [37m+ code: ASDC_4002,[0m [37m+ severity: ERROR,[0m [37m+ description: "Connection problem towards U-EB server. Cannot reach host %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeDistributionEngineInvalidArtifactType: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4006,[0m [37m+ severity: WARN,[0m [37m+ description: "The artifact type %s does not appear in the list of valid artifacts %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ BeInvalidTypeError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4008,[0m [37m+ severity: WARN,[0m [37m+ description: "The type %s of %s is invalid",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ BeInvalidValueError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_3028,[0m [37m+ severity: WARN,[0m [37m+ description: "The value %s of %s from type %s is invalid",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ } [0m [37m+ [0m [37m+ BeFailedDeletingResourceInstanceError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_3029,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to delete resource instance %s from service %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeMissingConfigurationError: {[0m [37m+ type: CONFIG_ERROR,[0m [37m+ code: ASDC_3030,[0m [37m+ severity: FATAL,[0m [37m+ description: "Configuration parameter %s is missing",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeConfigurationInvalidListSizeError: {[0m [37m+ type: CONFIG_ERROR,[0m [37m+ code: ASDC_3031,[0m [37m+ severity: FATAL,[0m [37m+ description: "Configuration parameter %s is invalid. At least %s values shall be configured",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ ErrorConfigFileFormat: {[0m [37m+ type: CONFIG_ERROR,[0m [37m+ code: ASDC_3032,[0m [37m+ severity: ERROR,[0m [37m+ description: "Error element not found in YAML name: %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeMissingArtifactInformationError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4010,[0m [37m+ severity: ERROR,[0m [37m+ description: "Artifact uploaded has missing information. Missing %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeArtifactMissingError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4011,[0m [37m+ severity: ERROR,[0m [37m+ description: "Artifact %s requested is not found",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeArtifactPayloadInvalid: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4012,[0m [37m+ severity: ERROR,[0m [37m+ description: "Payload of artifact uploaded is invalid (invalid MD5 or encryption)",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeUserMissingError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4009,[0m [37m+ severity: ERROR,[0m [37m+ description: "User %s requested is not found",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeArtifactInformationInvalidError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4013,[0m [37m+ severity: ERROR,[0m [37m+ description: "Input for artifact metadata is invalid",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ BeFailedAddingCapabilityTypeError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4015,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed adding capability type",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+ [0m [37m+ BeCapabilityTypeMissingError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4016,[0m [37m+ severity: ERROR,[0m [37m+ description: "Capability Type %s not found",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+ [0m [37m+ BeInterfaceMissingError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4020,[0m [37m+ severity: ERROR,[0m [37m+ description: "Interface %s required is missing",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeDaoSystemError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_4014,[0m [37m+ severity: ERROR,[0m [37m+ description: "Operation towards database failed",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+ [0m [37m+ BeSystemError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_4017,[0m [37m+ severity: ERROR,[0m [37m+ description: "Unexpected error during operation",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedLockObjectError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_4007,[0m [37m+ severity: WARN,[0m [37m+ description: "Failed to lock object for update",[0m [37m+ alarmSeverity: CRITICAL[0m [37m+ }[0m [37m+ [0m [37m+ BeInvalidJsonInput: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_4018,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to convert json input to object",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeDistributionMissingError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_4021,[0m [37m+ severity: ERROR,[0m [37m+ description: "Distribution %s required is missing",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeHealthCheckRecovery: {[0m [37m+ type: RECOVERY,[0m [37m+ code: ASDC_4022,[0m [37m+ severity: INFO,[0m [37m+ description: "BE Health Check Recovery",[0m [37m+ alarmSeverity: INFORMATIONAL[0m [37m+ }[0m [37m+ BeFailedCreateNodeError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6000,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to create node %s on graph. status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ BeFailedUpdateNodeError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6001,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to update node %s on graph. Status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedDeleteNodeError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6002,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to delete node %s on graph. Status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedRetrieveNodeError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6003,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to retrieve node %s from graph. Status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeExecuteRollbackError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6004,[0m [37m+ severity: ERROR,[0m [37m+ description: "Going to execute rollback on graph.",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedFindParentError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6005,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to find parent node %s on graph. Status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedFindAllNodesError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6006,[0m [37m+ severity: ERROR,[0m [37m+ description: "Failed to fetch all nodes with type %s of parent node %s . Status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedFindAssociationError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6007,[0m [37m+ severity: ERROR,[0m [37m+ description: "Cannot find node with type %s associated with node %s . Status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m [37m+ BeFailedFindAssociationError: {[0m [37m+ type: DATA_ERROR,[0m [37m+ code: ASDC_6008,[0m [37m+ severity: ERROR,[0m [37m+ description: "Cannot find node with type %s associated with node %s . Status is %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ BeComponentCleanerSystemError: {[0m [37m+ type: SYSTEM_ERROR,[0m [37m+ code: ASDC_6009,[0m [37m+ severity: ERROR,[0m [37m+ description: "Error occured in Component Cleaner Task. Failed operation: %s",[0m [37m+ alarmSeverity: MAJOR[0m [37m+ }[0m [37m+ [0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] mode changed to 644 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/ecomp-error-configuration.yaml] mode changed to 644 [32m- change mode from '' to '0644'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] action create_if_missing[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] created file /var/lib/jetty/config/catalog-be/error-configuration.yaml [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] created file /var/lib/jetty/config/catalog-be/error-configuration.yaml [32m- create new file /var/lib/jetty/config/catalog-be/error-configuration.yaml[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] updated file contents /var/lib/jetty/config/catalog-be/error-configuration.yaml [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] updated file contents /var/lib/jetty/config/catalog-be/error-configuration.yaml [32m- update content in file /var/lib/jetty/config/catalog-be/error-configuration.yaml from none to ffcde1[0m [37m--- /var/lib/jetty/config/catalog-be/error-configuration.yaml 2021-05-28 00:05:52.548752124 +0000[0m [37m+++ /var/lib/jetty/config/catalog-be/.chef-error-configuration20210528-22-1ldslas.yaml 2021-05-28 00:05:52.548752124 +0000[0m [37m@@ -1,2432 +1,4864 @@[0m [37m+# Errors[0m [37m+errors:[0m [37m+ OK: {[0m [37m+ code: 200,[0m [37m+ message: "OK"[0m [37m+ }[0m [37m+ CREATED: {[0m [37m+ code: 201,[0m [37m+ message: "OK"[0m [37m+ }[0m [37m+ ACCEPTED: {[0m [37m+ code: 202,[0m [37m+ message: "Accepted"[0m [37m+ }[0m [37m+ NO_CONTENT: {[0m [37m+ code: 204,[0m [37m+ message: "No Content"[0m [37m+ }[0m [37m+ #--------POL4050-----------------------------[0m [37m+ NOT_ALLOWED: {[0m [37m+ code: 405,[0m [37m+ message: "Error: Method not allowed.",[0m [37m+ messageId: "POL4050"[0m [37m+ }[0m [37m+ #--------POL5000-----------------------------[0m [37m+ GENERAL_ERROR: {[0m [37m+ code: 500,[0m [37m+ message: "Error: Internal Server Error. Please try again later.",[0m [37m+ messageId: "POL5000"[0m [37m+ }[0m [37m+ #---------POL5001------------------------------[0m [37m+ MISSING_X_ECOMP_INSTANCE_ID: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Missing 'X-ECOMP-InstanceID' HTTP header.",[0m [37m+ messageId: "POL5001"[0m [37m+ }[0m [37m+ #---------POL5002------------------------------[0m [37m+ AUTH_REQUIRED: {[0m [37m+ code: 401 ,[0m [37m+ message: "Error: Authentication is required to use the API.",[0m [37m+ messageId: "POL5002"[0m [37m+ }[0m [37m+ #---------POL5003------------------------------[0m [37m+ AUTH_FAILED: {[0m [37m+ code: 403 ,[0m [37m+ message: "Error: Not authorized to use the API.",[0m [37m+ messageId: "POL5003"[0m [37m+ }[0m [37m+ #---------POL5004------------------------------[0m [37m+ MISSING_USER_ID: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Missing 'USER_ID' HTTP header.",[0m [37m+ messageId: "POL5004"[0m [37m+ }[0m [37m+ #---------POL5005------------------------------[0m [37m+ SESSION_EXPIRED: {[0m [37m+ code: 403 ,[0m [37m+ message: "Your session has expired. Please close the SDC tab and re-enter the SDC application.",[0m [37m+ messageId: "POL5005"[0m [37m+ }[0m [37m+ #---------SVC4000-----------------------------[0m [37m+ INVALID_CONTENT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content.",[0m [37m+ messageId: "SVC4000"[0m [37m+ }[0m [37m+ #---------SVC4002-----------------------------[0m [37m+ MISSING_INFORMATION: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Missing information.",[0m [37m+ messageId: "SVC4002"[0m [37m+ }[0m [37m+ #---------SVC4003------------------------------[0m [37m+ # %1 - Users's ID[0m [37m+ USER_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: User '%1' was not found.",[0m [37m+ messageId: "SVC4003"[0m [37m+ }[0m [37m+ #---------SVC4004-----------------------------[0m [37m+ # %1 - Users's email address[0m [37m+ INVALID_EMAIL_ADDRESS: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid email address '%1'.",[0m [37m+ messageId: "SVC4004"[0m [37m+ }[0m [37m+ #---------SVC4005------------------------------[0m [37m+ # %1 - role[0m [37m+ INVALID_ROLE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid role '%1'.",[0m [37m+ messageId: "SVC4005"[0m [37m+ }[0m [37m+ #---------SVC4006------------------------------[0m [37m+ # %1 - Users's USER_ID[0m [37m+ USER_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: User with '%1' ID already exists.",[0m [37m+ messageId: "SVC4006"[0m [37m+ }[0m [37m+ #---------SVC4007------------------------------[0m [37m+ DELETE_USER_ADMIN_CONFLICT: {[0m [37m+ code: 409,[0m [37m+ message: "Error: An administrator can only be deleted by another administrator.",[0m [37m+ messageId: "SVC4007"[0m [37m+ }[0m [37m+ #---------SVC4008-----------------------------[0m [37m+ # %1 - Users's userId[0m [37m+ INVALID_USER_ID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid userId '%1'.",[0m [37m+ messageId: "SVC4008"[0m [37m+ }[0m [37m+#---------SVC4009-----------------------------[0m [37m+ USER_DEFINED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: User Defined '%1'.",[0m [37m+ messageId: "SVC4009"[0m [37m+ }[0m [37m+ #---------SVC4049------------------------------[0m [37m+ # %1 - service/resource[0m [37m+ COMPONENT_MISSING_CONTACT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing %1 contact.",[0m [37m+ messageId: "SVC4049"[0m [37m+ }[0m [37m+ #---------SVC4050-----------------------------[0m [37m+ # %1 - Service/Resource/Additional parameter[0m [37m+ # %2 - service/resource/label name[0m [37m+ COMPONENT_NAME_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: %1 with name '%2' already exists.",[0m [37m+ messageId: "SVC4050"[0m [37m+ }[0m [37m+ #---------SVC4051------------------------------[0m [37m+ # %1 - resource/service[0m [37m+ COMPONENT_MISSING_CATEGORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing %1 category.",[0m [37m+ messageId: "SVC4051"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4052------------------------------[0m [37m+ COMPONENT_MISSING_TAGS: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. At least one tag has to be specified.",[0m [37m+ messageId: "SVC4052"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4053------------------------------[0m [37m+ # %1 - service/resource[0m [37m+ COMPONENT_MISSING_DESCRIPTION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing %1 description.",[0m [37m+ messageId: "SVC4053"[0m [37m+ }[0m [37m+ #---------SVC4054------------------------------[0m [37m+ # %1 - resource/service[0m [37m+ COMPONENT_INVALID_CATEGORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Value %1 for the field Category is invalid.",[0m [37m+ messageId: "SVC4054"[0m [37m+ }[0m [37m+ #---------SVC4055------------------------------[0m [37m+ MISSING_VENDOR_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing vendor name.",[0m [37m+ messageId: "SVC4055"[0m [37m+ }[0m [37m+ #---------SVC4056------------------------------[0m [37m+ MISSING_VENDOR_RELEASE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing vendor release.",[0m [37m+ messageId: "SVC4056"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4057------------------------------[0m [37m+ MISSING_DERIVED_FROM_TEMPLATE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing derived from template specification.",[0m [37m+ messageId: "SVC4057"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4058------------------------------[0m [37m+ # %1 - service/resource[0m [37m+ COMPONENT_MISSING_ICON: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing %1 icon.",[0m [37m+ messageId: "SVC4058"[0m [37m+ }[0m [37m+ #---------SVC4059------------------------------[0m [37m+ # %1 - service/resource[0m [37m+ COMPONENT_INVALID_ICON: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Invalid %1 icon.",[0m [37m+ messageId: "SVC4059"[0m [37m+ }[0m [37m+ #---------SVC4060------------------------------[0m [37m+ PARENT_RESOURCE_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Derived from resource template was not found.",[0m [37m+ messageId: "SVC4060"[0m [37m+ }[0m [37m+ #---------SVC4061------------------------------[0m [37m+ MULTIPLE_PARENT_RESOURCE_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Multiple derived from resource template is not allowed.",[0m [37m+ messageId: "SVC4061"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4062------------------------------[0m [37m+ # %1 - service/resource[0m [37m+ MISSING_COMPONENT_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing %1 name.",[0m [37m+ messageId: "SVC4062"[0m [37m+ }[0m [37m+ #---------SVC4063------------------------------[0m [37m+ #%1 - resource/service name[0m [37m+ RESOURCE_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' resource was not found.",[0m [37m+ messageId: "SVC4063"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4064------------------------------[0m [37m+ # %1 - Service/Resource/Property[0m [37m+ COMPONENT_INVALID_DESCRIPTION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. %1 description contains non-english characters.",[0m [37m+ messageId: "SVC4064"[0m [37m+ }[0m [37m+ #---------SVC4065------------------------------[0m [37m+ # %1 - Service/Resource/Property[0m [37m+ # %2 - max resource/service name length[0m [37m+ COMPONENT_DESCRIPTION_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. %1 description exceeds limit of %2 characters.",[0m [37m+ messageId: "SVC4065"[0m [37m+ }[0m [37m+ #---------SVC4066------------------------------[0m [37m+ # %1 - max length[0m [37m+ COMPONENT_TAGS_EXCEED_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Tags overall length exceeds limit of %1 characters.",[0m [37m+ messageId: "SVC4066"[0m [37m+ }[0m [37m+ #---------SVC4067------------------------------[0m [37m+ # %1 - max length[0m [37m+ VENDOR_NAME_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Vendor name exceeds limit of %1 characters.",[0m [37m+ messageId: "SVC4067"[0m [37m+ }[0m [37m+ #---------SVC4068------------------------------[0m [37m+ # %1 - max length[0m [37m+ VENDOR_RELEASE_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Vendor release exceeds limit of %1 characters.",[0m [37m+ messageId: "SVC4068"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4069------------------------------[0m [37m+ # %1 - Service/Resource/Product[0m [37m+ COMPONENT_INVALID_CONTACT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. %1 Contact Id should be in format 'mnnnnnn' or 'aannna' or 'aannnn', where m=m ,a=a-zA-Z and n=0-9. Max length: 50",[0m [37m+ messageId: "SVC4069"[0m [37m+ }[0m [37m+ #---------SVC4070------------------------------[0m [37m+ # %1 - Service/Resource[0m [37m+ INVALID_COMPONENT_NAME: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. Value %1 for Component name is invalid. Only alphanumeric characters and regular spaces are allowed.',[0m [37m+ messageId: "SVC4070"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4071------------------------------[0m [37m+ INVALID_VENDOR_NAME: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. Value %1 for the field Vendor name is invalid. Only alphanumeric characters and regular spaces are allowed.',[0m [37m+ messageId: "SVC4071"[0m [37m+ }[0m [37m+ #---------SVC4072------------------------------[0m [37m+ INVALID_VENDOR_RELEASE: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. Value %1 for the field Vendor release is invalid. Only alphanumeric characters and regular spaces are allowed.',[0m [37m+ messageId: "SVC4072"[0m [37m+ }[0m [37m+ #---------SVC4073------------------------------[0m [37m+ # %1 - Service/Resource[0m [37m+ # %2 - max resource/service name[0m [37m+ COMPONENT_NAME_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. %1 name exceeds limit of %2 characters.",[0m [37m+ messageId: "SVC4073"[0m [37m+ }[0m [37m+ #---------SVC4080------------------------------[0m [37m+ # %1 - resource/service name[0m [37m+ # %2 - resource/service[0m [37m+ # %3 - First name of last modifier[0m [37m+ # %4 - Last name of last modifier[0m [37m+ # %5 - USER_ID of last modifier[0m [37m+ COMPONENT_IN_CHECKOUT_STATE: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested '%1' %2 is locked for modification by %3 %4(%5).",[0m [37m+ messageId: "SVC4080"[0m [37m+ }[0m [37m+ #---------SVC4081-----------------------------[0m [37m+ # %1 - resource/service name[0m [37m+ # %2 - resource/service[0m [37m+ # %3 - First name of last modifier[0m [37m+ # %4 - Last name of last modifier[0m [37m+ # %5 - USER_ID of last modifier[0m [37m+ COMPONENT_IN_CERT_IN_PROGRESS_STATE: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested '%1' %2 is locked for certification by %3 %4(%5).",[0m [37m+ messageId: "SVC4081"[0m [37m+ }[0m [37m+[0m [37m+ #-----------SVC4082---------------------------[0m [37m+ # %1 - resource/service name[0m [37m+ # %2 - resource/service[0m [37m+ # %3 - First name of last modifier[0m [37m+ # %4 - Last name of last modifier[0m [37m+ # %5 - USER_ID of last modifier[0m [37m+ COMPONENT_SENT_FOR_CERTIFICATION: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested '%1' %2 is sent for certification by %3 %4(%5).",[0m [37m+ messageId: "SVC4082"[0m [37m+ }[0m [37m+ #-----------SVC4083---------------------------[0m [37m+ COMPONENT_VERSION_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Version of this %1 was already promoted.",[0m [37m+ messageId: "SVC4083"[0m [37m+ }[0m [37m+ #-----------SVC4084---------------------------[0m [37m+ # %1 - resource/service/product name[0m [37m+ # %2 - resource/service/product[0m [37m+ # %3 - First name of last modifier[0m [37m+ # %4 - Last name of last modifier[0m [37m+ # %5 - USER_ID of last modifier[0m [37m+ COMPONENT_ALREADY_CHECKED_IN: {[0m [37m+ code: 409,[0m [37m+ message: "Error: The current version of '%1' %2 was already checked-in by %3 %4(%5).",[0m [37m+ messageId: "SVC4084"[0m [37m+ }[0m [37m+ #-----------SVC4085---------------------------[0m [37m+ # %1 - resource/service/product name[0m [37m+ # %2 - resource/service/product[0m [37m+ # %3 - First name of last modifier[0m [37m+ # %4 - Last name of last modifier[0m [37m+ # %5 - USER_ID of last modifier[0m [37m+ COMPONENT_CHECKOUT_BY_ANOTHER_USER: {[0m [37m+ code: 403,[0m [37m+ message: "Error: %1 %2 has already been checked out by %3 %4(%5).",[0m [37m+ messageId: "SVC4085"[0m [37m+ }[0m [37m+ #-----------SVC4086---------------------------[0m [37m+ # %1 - resource/service name[0m [37m+ # %2 - resource/service[0m [37m+ COMPONENT_IN_USE: {[0m [37m+ code: 403,[0m [37m+ message: "Error: '%1' %2 is still held by previous action. Please try again later.",[0m [37m+ messageId: "SVC4086"[0m [37m+ }[0m [37m+ #-----------SVC4087---------------------------[0m [37m+ # %1 - component name[0m [37m+ # %2 - resource/service/product[0m [37m+ COMPONENT_HAS_NEWER_VERSION: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Checking out of the requested version of the '%1' %2 is not allowed as a newer version exists.",[0m [37m+ messageId: "SVC4087"[0m [37m+ }[0m [37m+ #-----------SVC4088---------------------------[0m [37m+ # %1 - resource/service name[0m [37m+ # %2 - resource/service[0m [37m+ # %3 - First name of last modifier[0m [37m+ # %4 - Last name of last modifier[0m [37m+ # %5 - USER_ID of last modifier[0m [37m+ COMPONENT_ALREADY_CERTIFIED: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested %1 %2 has already been certified by %3 %4(%5).",[0m [37m+ messageId: "SVC4088"[0m [37m+ }[0m [37m+ #-----------SVC4089---------------------------[0m [37m+ # %1 - resource/service name[0m [37m+ # %2 - resource/service[0m [37m+ COMPONENT_NOT_READY_FOR_CERTIFICATION: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested '%1' %2 is not ready for certification.",[0m [37m+ messageId: "SVC4089"[0m [37m+ }[0m [37m+ #-----------SVC4100---------------------------[0m [37m+ #%1 - property name[0m [37m+ PROPERTY_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' property was not found.",[0m [37m+ messageId: "SVC4100"[0m [37m+ }[0m [37m+ #-----------SVC4101---------------------------[0m [37m+ #%1 - property name[0m [37m+ PROPERTY_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Property with '%1' name already exists.",[0m [37m+ messageId: "SVC4101"[0m [37m+ }[0m [37m+[0m [37m+ #-----------SVC4102---------------------------[0m [37m+ # %1 - capability type name[0m [37m+ CAPABILITY_TYPE_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Capability Type with name '%1' already exists.",[0m [37m+ messageId: "SVC4102"[0m [37m+ }[0m [37m+ #-----------SVC4114---------------------------[0m [37m+ AUTH_FAILED_INVALIDE_HEADER: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Authorization header.",[0m [37m+ messageId: "SVC4114"[0m [37m+ }[0m [37m+ #-----------SVC4115---------------------------[0m [37m+ # %1 - capability type name[0m [37m+ MISSING_CAPABILITY_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing Capability Type '%1'.",[0m [37m+ messageId: "SVC4115"[0m [37m+ }[0m [37m+ RESOURCE_INSTANCE_BAD_REQUEST: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content.",[0m [37m+ messageId: "SVC4116"[0m [37m+ }[0m [37m+ #-----------SVC4117---------------------------[0m [37m+ # %1 - resource instance name[0m [37m+ # %2 - resource instance name[0m [37m+ # %3 - requirement name[0m [37m+ RESOURCE_INSTANCE_MATCH_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Match not found between resource instance '%1' and resource instance '%2' for requirement '%3'.",[0m [37m+ messageId: "SVC4117"[0m [37m+ }[0m [37m+ #-----------SVC4118---------------------------[0m [37m+ # %1 - resource instance name[0m [37m+ # %2 - resource instance name[0m [37m+ # %3 - requirement name[0m [37m+ RESOURCE_INSTANCE_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Resource instances '%1' and '%2' are already associated with requirement '%3'.",[0m [37m+ messageId: "SVC4118"[0m [37m+ }[0m [37m+ #-----------SVC4119---------------------------[0m [37m+ # %1 - resource instance name[0m [37m+ # %2 - resource instance name[0m [37m+ # %3 - requirement name[0m [37m+ RESOURCE_INSTANCE_RELATION_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: No relation found between resource instances '%1' and '%2' for requirement '%3'.",[0m [37m+ messageId: "SVC4119"[0m [37m+ }[0m [37m+ #-----------SVC4120---------------------------[0m [37m+ # %1 - User's USER_ID[0m [37m+ USER_INACTIVE: {[0m [37m+ code: 404,[0m [37m+ message: "Error: User %1 inactive.",[0m [37m+ messageId: "SVC4120"[0m [37m+ }[0m [37m+ #-----------SVC4121---------------------------[0m [37m+ # %1 - User's USER_ID[0m [37m+ USER_HAS_ACTIVE_ELEMENTS: {[0m [37m+ code: 403,[0m [37m+ message: "Error: User with %1 ID can not be deleted since it has active elements(resources/services/artifacts).",[0m [37m+ messageId: "SVC4121"[0m [37m+ }[0m [37m+ #-----------SVC4122---------------------------[0m [37m+ # %1 - artifact type[0m [37m+ ARTIFACT_TYPE_NOT_SUPPORTED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid artifact type '%1'.",[0m [37m+ messageId: "SVC4122"[0m [37m+ }[0m [37m+ #-----------SVC4123---------------------------[0m [37m+ ARTIFACT_LOGICAL_NAME_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Artifact logical name cannot be changed.",[0m [37m+ messageId: "SVC4123"[0m [37m+ }[0m [37m+ #-----------SVC4124---------------------------[0m [37m+ MISSING_ARTIFACT_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Missing artifact type.",[0m [37m+ messageId: "SVC4124"[0m [37m+ }[0m [37m+ #-----------SVC4125---------------------------[0m [37m+ # %1-artifact name[0m [37m+ ARTIFACT_EXIST: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Artifact '%1' already exists.",[0m [37m+ messageId: "SVC4125"[0m [37m+ }[0m [37m+ #---------SVC4126------------------------------[0m [37m+ # %1 - resource/service/product/...[0m [37m+ # %2 - field (tag, vendor name...)[0m [37m+ INVALID_FIELD_FORMAT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid %1 %2 format.",[0m [37m+ messageId: "SVC4126"[0m [37m+ }[0m [37m+ #-----------SVC4127---------------------------[0m [37m+ ARTIFACT_INVALID_MD5: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid artifact checksum.",[0m [37m+ messageId: "SVC4127"[0m [37m+ }[0m [37m+ #-----------SVC4128---------------------------[0m [37m+ MISSING_ARTIFACT_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. Missing artifact name.",[0m [37m+ messageId: "SVC4128"[0m [37m+ }[0m [37m+ #-----------SVC4129---------------------------[0m [37m+ MISSING_PROJECT_CODE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing PROJECT_CODE number.",[0m [37m+ messageId: "SVC4129"[0m [37m+ }[0m [37m+ #-----------SVC4130---------------------------[0m [37m+ INVALID_PROJECT_CODE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. PROJECT_CODE must be from 5 up to 50 characters.",[0m [37m+ messageId: "SVC4130"[0m [37m+ }[0m [37m+ #-----------SVC4131---------------------------[0m [37m+ # %1-resource/service[0m [37m+ # %2-srtifact/artifacts[0m [37m+ # %3-semicolomn separated list of artifact[0m [37m+ COMPONENT_MISSING_MANDATORY_ARTIFACTS: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Missing mandatory informational %1 %2: [%3].",[0m [37m+ messageId: "SVC4131"[0m [37m+ }[0m [37m+ #-----------SVC4132---------------------------[0m [37m+ # %1 - lifecycle type name[0m [37m+ LIFECYCLE_TYPE_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Lifecycle Type with name '%1' already exists.",[0m [37m+ messageId: "SVC4132"[0m [37m+ }[0m [37m+ #-----------SVC4133---------------------------[0m [37m+ # %1 - service version[0m [37m+ # %2 - service name[0m [37m+ SERVICE_NOT_AVAILABLE_FOR_DISTRIBUTION: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Version %1 of '%2' service is not available for distribution.",[0m [37m+ messageId: "SVC4133"[0m [37m+ }[0m [37m+ #-----------SVC4134---------------------------[0m [37m+ MISSING_LIFECYCLE_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing interface life-cycle type.",[0m [37m+ messageId: "SVC4134"[0m [37m+ }[0m [37m+ #---------SVC4135------------------------------[0m [37m+ SERVICE_CATEGORY_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Service category cannot be changed once the service is certified.",[0m [37m+ messageId: "SVC4135"[0m [37m+ }[0m [37m+ #---------SVC4136------------------------------[0m [37m+ # %1 - distribution environment name[0m [37m+ DISTRIBUTION_ENVIRONMENT_NOT_AVAILABLE: {[0m [37m+ code: 500,[0m [37m+ message: "Error: Requested distribution environment '%1' is not available.",[0m [37m+ messageId: "SVC4136"[0m [37m+ }[0m [37m+ #---------SVC4137------------------------------[0m [37m+ # %1 - distribution environment name[0m [37m+ DISTRIBUTION_ENVIRONMENT_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Requested distribution environment '%1' was not found.",[0m [37m+ messageId: "SVC4137"[0m [37m+ }[0m [37m+ #---------SVC4138------------------------------[0m [37m+ DISTRIBUTION_ENVIRONMENT_INVALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid distribution environment.",[0m [37m+ messageId: "SVC4138"[0m [37m+ }[0m [37m+ #---------SVC4200------------------------------[0m [37m+ # %1 - Service/Resource[0m [37m+ # %2 - max icon name length[0m [37m+ COMPONENT_ICON_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. %1 icon name exceeds limit of %2 characters.",[0m [37m+ messageId: "SVC4200"[0m [37m+ }[0m [37m+ #---------SVC4300------------------------------[0m [37m+ RESTRICTED_ACCESS: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Restricted access.",[0m [37m+ messageId: "SVC4300"[0m [37m+ }[0m [37m+ #---------SVC4301------------------------------[0m [37m+ RESTRICTED_OPERATION: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Restricted operation.",[0m [37m+ messageId: "SVC4301"[0m [37m+ }[0m [37m+ #---------SVC4500------------------------------[0m [37m+ MISSING_BODY: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Missing request body.",[0m [37m+ messageId: "SVC4500"[0m [37m+ }[0m [37m+ #---------SVC4501------------------------------[0m [37m+ MISSING_PUBLIC_KEY: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Invalid Content. Missing mandatory parameter 'apiPublicKey'." ,[0m [37m+ messageId: "SVC4501"[0m [37m+ }[0m [37m+ #---------SVC4502------------------------------[0m [37m+ DISTRIBUTION_ENV_DOES_NOT_EXIST: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Invalid Body : Missing mandatory parameter 'distrEnvName'." ,[0m [37m+ messageId: "SVC4502"[0m [37m+ }[0m [37m+ #-----------SVC4503---------------------------[0m [37m+ # %1 - service name[0m [37m+ SERVICE_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' service was not found.",[0m [37m+ messageId: "SVC4503"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4504------------------------------[0m [37m+ # %1 - Service/Resource[0m [37m+ # %2 - service/resource version[0m [37m+ COMPONENT_VERSION_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: %1 version %2 was not found.",[0m [37m+ messageId: "SVC4504"[0m [37m+ }[0m [37m+ #-----------SVC4505---------------------------[0m [37m+ #%1-artifact name[0m [37m+[0m [37m+ ARTIFACT_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Artifact '%1' was not found.",[0m [37m+ messageId: "SVC4505"[0m [37m+ }[0m [37m+ #---------SVC4506------------------------------[0m [37m+ MISSING_ENV_NAME: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Invalid Content. Missing mandatory parameter 'distrEnvName'.",[0m [37m+ messageId: "SVC4506"[0m [37m+ }[0m [37m+ #---------SVC4507------------------------------[0m [37m+ COMPONENT_INVALID_TAGS_NO_COMP_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. One of the tags should be the component name.",[0m [37m+ messageId: "SVC4507"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4508------------------------------[0m [37m+ SERVICE_NAME_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Service name cannot be changed once the service is certified.",[0m [37m+ messageId: "SVC4508"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4509------------------------------[0m [37m+ SERVICE_ICON_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Icon cannot be changed once the service is certified.",[0m [37m+ messageId: "SVC4509"[0m [37m+ }[0m [37m+ #---------SVC4510------------------------------[0m [37m+ # %1 - icon name max length[0m [37m+ SERVICE_ICON_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Icon name exceeds limit of %1 characters.",[0m [37m+ messageId: "SVC4510"[0m [37m+ }[0m [37m+ #---------SVC4511------------------------------[0m [37m+ DISTRIBUTION_REQUESTED_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested distribution was not found.",[0m [37m+ messageId: "SVC4511"[0m [37m+ }[0m [37m+ #---------SVC4512------------------------------[0m [37m+ # %1 - Distribution ID[0m [37m+ DISTRIBUTION_REQUESTED_FAILED: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested distribution '%1' failed.",[0m [37m+ messageId: "SVC4512"[0m [37m+ }[0m [37m+ #---------SVC4513------------------------------[0m [37m+ RESOURCE_CATEGORY_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Resource category cannot be changed once the resource is certified.",[0m [37m+ messageId: "SVC4513"[0m [37m+ }[0m [37m+ #---------SVC4514------------------------------[0m [37m+ RESOURCE_NAME_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Resource name cannot be changed once the resource is certified.",[0m [37m+ messageId: "SVC4514"[0m [37m+ }[0m [37m+ #---------SVC4515------------------------------[0m [37m+ RESOURCE_ICON_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Icon cannot be changed once the resource is certified.",[0m [37m+ messageId: "SVC4515"[0m [37m+ }[0m [37m+ #---------SVC4516------------------------------[0m [37m+ RESOURCE_VENDOR_NAME_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Vendor name cannot be changed once the resource is certified.",[0m [37m+ messageId: "SVC4516"[0m [37m+ }[0m [37m+ #---------SVC4517------------------------------[0m [37m+ RESOURCE_DERIVED_FROM_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Derived from resource template cannot be changed once the resource is certified.",[0m [37m+ messageId: "SVC4517"[0m [37m+ }[0m [37m+ #---------SVC4518------------------------------[0m [37m+ # %1 - max length[0m [37m+ COMPONENT_SINGLE_TAG_EXCEED_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Single tag exceeds limit of %1 characters.",[0m [37m+ messageId: "SVC4518"[0m [37m+ }[0m [37m+ #---------SVC4519------------------------------[0m [37m+ INVALID_DEFAULT_VALUE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: mismatch in data-type occurred for property %1. data type is %2 and default value found is %3.",[0m [37m+ messageId: "SVC4519"[0m [37m+ }[0m [37m+ #---------SVC4520------------------------------[0m [37m+ # %1 - service or resource[0m [37m+ ADDITIONAL_INFORMATION_MAX_NUMBER_REACHED: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Maximal number of additional %1 parameters was reached.",[0m [37m+ messageId: "SVC4520"[0m [37m+ }[0m [37m+ #---------SVC4521------------------------------[0m [37m+ ADDITIONAL_INFORMATION_EMPTY_STRING_NOT_ALLOWED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. The Additional information label and value cannot be empty.",[0m [37m+ messageId: "SVC4521"[0m [37m+ }[0m [37m+ #---------SVC4522------------------------------[0m [37m+ # %1 - label/value[0m [37m+ # %2 - Maximal length of %1[0m [37m+ ADDITIONAL_INFORMATION_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Additional information %1 exceeds limit of %2 characters.",[0m [37m+ messageId: "SVC4522"[0m [37m+ }[0m [37m+ #---------SVC4523------------------------------[0m [37m+ ADDITIONAL_INFORMATION_KEY_NOT_ALLOWED_CHARACTERS: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. Additional information label is not allowed to contain characters like <>:"\/|?* and space characters other than regular space.',[0m [37m+ messageId: "SVC4523"[0m [37m+ }[0m [37m+ #---------SVC4524------------------------------[0m [37m+ ADDITIONAL_INFORMATION_NOT_FOUND: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Requested additional information was not found.",[0m [37m+ messageId: "SVC4524"[0m [37m+ }[0m [37m+ #---------SVC4525------------------------------[0m [37m+ ADDITIONAL_INFORMATION_VALUE_NOT_ALLOWED_CHARACTERS: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. Additional information contains non-english characters.',[0m [37m+ messageId: "SVC4525"[0m [37m+ }[0m [37m+ #---------SVC4526------------------------------[0m [37m+ RESOURCE_INSTANCE_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' resource instance was not found.",[0m [37m+ messageId: "SVC4526"[0m [37m+ }[0m [37m+ #---------SVC4527------------------------------[0m [37m+ ASDC_VERSION_NOT_FOUND: {[0m [37m+ code: 500,[0m [37m+ message: 'Error: ASDC version cannot be displayed.',[0m [37m+ messageId: "SVC4527"[0m [37m+ }[0m [37m+ #---------SVC4528------------------------------[0m [37m+ # %1-artifact url/artifact label/artifact description/VNF Service Indicator[0m [37m+ MISSING_DATA: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. Missing %1.",[0m [37m+ messageId: "SVC4528"[0m [37m+ }[0m [37m+ #---------SVC4529------------------------------[0m [37m+ # %1-artifact url/artifact label/artifact description/artifact name[0m [37m+ # %2 - Maximal length of %1[0m [37m+ EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. %1 exceeds limit of %2 characters.",[0m [37m+ messageId: "SVC4529"[0m [37m+ }[0m [37m+ #---------SVC4530------------------------------[0m [37m+ ARTIFACT_INVALID_TIMEOUT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Artifact Timeout should be set to valid positive non-zero number of minutes.",[0m [37m+ messageId: "SVC4530"[0m [37m+ }[0m [37m+ #---------SVC4531------------------------------[0m [37m+ SERVICE_IS_VNF_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: VNF Indicator cannot be updated for certified service.",[0m [37m+ messageId: "SVC4531"[0m [37m+ }[0m [37m+ #---------SVC4532------------------------------[0m [37m+ RESOURCE_INSTANCE_NOT_FOUND_ON_SERVICE: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' resource instance was not found on the service '%2.",[0m [37m+ messageId: "SVC4532"[0m [37m+ }[0m [37m+ #---------SVC4533------------------------------[0m [37m+ # %1 - "HEAT"/"HEAT_ENV"/"MURANO_PKG"/"YANG_XML"[0m [37m+ WRONG_ARTIFACT_FILE_EXTENSION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid file extension for %1 artifact type.",[0m [37m+ messageId: "SVC4533"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4534------------------------------[0m [37m+ # %1 - "HEAT"/"HEAT_ENV"[0m [37m+ INVALID_YAML: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Uploaded YAML file for %1 artifact is invalid.",[0m [37m+ messageId: "SVC4534"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4535------------------------------[0m [37m+ # %1 - "HEAT"[0m [37m+ INVALID_DEPLOYMENT_ARTIFACT_HEAT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid %1 artifact.",[0m [37m+ messageId: "SVC4535"[0m [37m+ }[0m [37m+ #---------SVC4536------------------------------[0m [37m+ # %1 - "Resource"/"Service"[0m [37m+ # %2 - resource/service name[0m [37m+ # %3 - "HEAT"/"HEAT_ENV"/"MURANO_PKG"[0m [37m+ # %4 - "HEAT"/"HEAT_ENV"/"MURANO_PKG[0m [37m+ DEPLOYMENT_ARTIFACT_OF_TYPE_ALREADY_EXISTS: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 '%2' already has a deployment artifact of %3 type .Please delete or update an existing %4 artifact.",[0m [37m+ messageId: "SVC4536"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4537------------------------------[0m [37m+ MISSING_HEAT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Missing HEAT artifact. HEAT_ENV artifact cannot be uploaded without corresponding HEAT template.",[0m [37m+ messageId: "SVC4537"[0m [37m+ }[0m [37m+ #---------SVC4538------------------------------[0m [37m+ MISMATCH_HEAT_VS_HEAT_ENV: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid artifact content. Parameter's set in HEAT_ENV '%1' artifact doesn't match the parameters in HEAT '%2' artifact.",[0m [37m+ messageId: "SVC4538"[0m [37m+ }[0m [37m+ #---------SVC4539------------------------------[0m [37m+ INVALID_RESOURCE_PAYLOAD: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid resource payload.",[0m [37m+ messageId: "SVC4539"[0m [37m+ }[0m [37m+ #---------SVC4540------------------------------[0m [37m+ INVALID_TOSCA_FILE_EXTENSION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid file extension for TOSCA template.",[0m [37m+ messageId: "SVC4540"[0m [37m+ }[0m [37m+ #---------SVC4541------------------------------[0m [37m+ INVALID_YAML_FILE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid YAML file.",[0m [37m+ messageId: "SVC4541"[0m [37m+ }[0m [37m+ #---------SVC4542------------------------------[0m [37m+ INVALID_TOSCA_TEMPLATE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid TOSCA template.",[0m [37m+ messageId: "SVC4542"[0m [37m+ }[0m [37m+ #---------SVC4543------------------------------[0m [37m+ NOT_RESOURCE_TOSCA_TEMPLATE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Imported Service TOSCA template.",[0m [37m+ messageId: "SVC4543"[0m [37m+ }[0m [37m+ #---------SVC4544------------------------------[0m [37m+ NOT_SINGLE_RESOURCE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Imported TOSCA template should contain one resource definition.",[0m [37m+ messageId: "SVC4544"[0m [37m+ }[0m [37m+ #---------SVC4545------------------------------[0m [37m+ INVALID_RESOURCE_NAMESPACE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid resource namespace.",[0m [37m+ messageId: "SVC4545"[0m [37m+ }[0m [37m+ #---------SVC4546------------------------------[0m [37m+ RESOURCE_ALREADY_EXISTS: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Imported resource already exists in ASDC Catalog.",[0m [37m+ messageId: "SVC4546"[0m [37m+ }[0m [37m+ #---------SVC4549------------------------------[0m [37m+ INVALID_RESOURCE_CHECKSUM: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid resource checksum.",[0m [37m+ messageId: "SVC4549"[0m [37m+ }[0m [37m+ #---------SVC4551------------------------------[0m [37m+ # %1 - ECOMP User name[0m [37m+ ECOMP_USER_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: ECOMP User '%1' was not found.",[0m [37m+ messageId: "SVC4551"[0m [37m+ }[0m [37m+ #---------SVC4554------------------------------[0m [37m+ # %1 - Artifact Id[0m [37m+ # %2 - "Resource"/"Service"[0m [37m+ COMPONENT_ARTIFACT_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested artifact %1 doesn't belong to specified %2.",[0m [37m+ messageId: "SVC4554"[0m [37m+ }[0m [37m+ #---------SVC4554------------------------------[0m [37m+ # %1 - "Service name"[0m [37m+ SERVICE_DEPLOYMENT_ARTIFACT_NOT_FOUND: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested '%1' service is not ready for certification. Service has to have at least one deployment artifact.",[0m [37m+ messageId: "SVC4554"[0m [37m+ }[0m [37m+ #---------SVC4555------------------------------[0m [37m+ # %1 - "Resource"/"Service"/"Product"[0m [37m+ # %2 - "category"[0m [37m+ COMPONENT_ELEMENT_INVALID_NAME_LENGTH: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid %1 %2 name length.",[0m [37m+ messageId: "SVC4555"[0m [37m+ }[0m [37m+ #---------SVC4556------------------------------[0m [37m+ # %1 - "Resource"/"Service"/"Product"[0m [37m+ # %2 - "category"[0m [37m+ COMPONENT_ELEMENT_INVALID_NAME_FORMAT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid %1 %2 name format.",[0m [37m+ messageId: "SVC4556"[0m [37m+ }[0m [37m+ #---------SVC4557------------------------------[0m [37m+ # %1 - "Resource"/"Service"/"Product"[0m [37m+ # %2 - "category name"[0m [37m+ COMPONENT_CATEGORY_ALREADY_EXISTS: {[0m [37m+ code: 409,[0m [37m+ message: "Error: %1 category name '%2' already exists.",[0m [37m+ messageId: "SVC4557"[0m [37m+ }[0m [37m+ #---------SVC4558------------------------------[0m [37m+ # %1 - "service"/"VF"[0m [37m+ # %2 - "Resource name"[0m [37m+ VALIDATED_RESOURCE_NOT_FOUND: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Submit for Testing is not permitted as your '%1' includes non-validated '%2' resource.",[0m [37m+ messageId: "SVC4558"[0m [37m+ }[0m [37m+ #---------SVC4559------------------------------[0m [37m+ # %1 - "service"/"VF"[0m [37m+ # %2 - "Resource name"[0m [37m+ FOUND_ALREADY_VALIDATED_RESOURCE: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Submit for Testing is not permitted as your '%1' includes non-validated '%2' resource. Please use already available validated resource version.",[0m [37m+ messageId: "SVC4559"[0m [37m+ }[0m [37m+ #---------SVC4560------------------------------[0m [37m+ # %1 - "service"/"VF"[0m [37m+ # %2 - "Resource name"[0m [37m+ FOUND_LIST_VALIDATED_RESOURCES: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Submit for Testing is not permitted as your '%1' includes non-validated '%2' resource. Please use one of available validated resource versions.",[0m [37m+ messageId: "SVC4560"[0m [37m+ }[0m [37m+ #---------SVC4561------------------------------[0m [37m+ # %1 - "resource"/"product"[0m [37m+ # %2 - "category"[0m [37m+ # %3 - "category name"[0m [37m+ COMPONENT_CATEGORY_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested %1 %2 '%3' was not found.",[0m [37m+ messageId: "SVC4561"[0m [37m+ }[0m [37m+ #---------SVC4562------------------------------[0m [37m+ # %1 - "Resource"/"Product"[0m [37m+ # %2 - "sub-category name"[0m [37m+ # %3 - "category name"[0m [37m+ COMPONENT_SUB_CATEGORY_EXISTS_FOR_CATEGORY: {[0m [37m+ code: 409,[0m [37m+ message: "Error: %1 sub-category '%2' already exists under '%3' category.",[0m [37m+ messageId: "SVC4562"[0m [37m+ }[0m [37m+ #---------SVC4563------------------------------[0m [37m+ # %1 - "Product"[0m [37m+ # %2 - "grouping name"[0m [37m+ # %3 - "sub-category name"[0m [37m+ COMPONENT_GROUPING_EXISTS_FOR_SUB_CATEGORY: {[0m [37m+ code: 409,[0m [37m+ message: "Error: %1 grouping '%2' already exists under '%3' sub-category.",[0m [37m+ messageId: "SVC4563"[0m [37m+ }[0m [37m+ #---------SVC4564------------------------------[0m [37m+ # %1 - product name[0m [37m+ PRODUCT_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' product was not found.",[0m [37m+ messageId: "SVC4564"[0m [37m+ }[0m [37m+ #---------SVC4565------------------------------[0m [37m+ # %1 - "HEAT"[0m [37m+ # %2 - parameter type ("string" , "boolean" , "number")[0m [37m+ # %3 - parameter name[0m [37m+ INVALID_HEAT_PARAMETER_VALUE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid %1 artifact. Invalid %2 value set for '%3' parameter.",[0m [37m+ messageId: "SVC4565"[0m [37m+ }[0m [37m+ #---------SVC4566------------------------------[0m [37m+ # %1 - "HEAT"[0m [37m+ # %2 - parameter type ("string" , "boolean" , "number")[0m [37m+ INVALID_HEAT_PARAMETER_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid %1 artifact. Unsupported '%2' parameter type.",[0m [37m+ messageId: "SVC4566"[0m [37m+ }[0m [37m+ #---------SVC4567------------------------------[0m [37m+ # %1 - "YANG_XML"[0m [37m+ INVALID_XML: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Uploaded XML file for %1 artifact is invalid.",[0m [37m+ messageId: "SVC4567"[0m [37m+ }[0m [37m+ #---------SVC4569------------------------------[0m [37m+ # %1 - "User Name and UserId"[0m [37m+ # %2 -"component names ot IDs"[0m [37m+ CANNOT_DELETE_USER_WITH_ACTIVE_ELEMENTS: {[0m [37m+ code: 412,[0m [37m+ message: "Error: User cannot be deleted. User '%1' has projects that cannot be committed: %2.",[0m [37m+ messageId: "SVC4569"[0m [37m+ }[0m [37m+ #---------SVC4568------------------------------[0m [37m+ # %1 - "User Name and UserId"[0m [37m+ # %2 -"checked-out"/"in-certification"[0m [37m+ CANNOT_UPDATE_USER_WITH_ACTIVE_ELEMENTS: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Role cannot be changed. User '%1' has %2 projects.",[0m [37m+ messageId: "SVC4568"[0m [37m+ }[0m [37m+ #---------SVC4570------------------------------[0m [37m+ UPDATE_USER_ADMIN_CONFLICT: {[0m [37m+ code: 409,[0m [37m+ message: "Error: An administrator is not allowed to change his/her role.",[0m [37m+ messageId: "SVC4570"[0m [37m+ }[0m [37m+ #---------SVC4571------------------------------[0m [37m+ SERVICE_CANNOT_CONTAIN_SUBCATEGORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Sub category cannot be defined for service",[0m [37m+ messageId: "SVC4571"[0m [37m+ }[0m [37m+ #---------SVC4572------------------------------[0m [37m+ # %1 - "Resource"/"Service"[0m [37m+ COMPONENT_TOO_MUCH_CATEGORIES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 must have only 1 category",[0m [37m+ messageId: "SVC4572"[0m [37m+ }[0m [37m+ #---------SVC4574------------------------------[0m [37m+ RESOURCE_TOO_MUCH_SUBCATEGORIES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Resource must have only 1 sub category",[0m [37m+ messageId: "SVC4574"[0m [37m+ }[0m [37m+ #---------SVC4575------------------------------[0m [37m+ COMPONENT_MISSING_SUBCATEGORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Missing sub category",[0m [37m+ messageId: "SVC4575"[0m [37m+ }[0m [37m+ #---------SVC4576------------------------------[0m [37m+ # %1 - "component type"[0m [37m+ UNSUPPORTED_ERROR: {[0m [37m+ code: 400,[0m [37m+ message: "Error : Requested component type %1 is unsupported.",[0m [37m+ messageId: "SVC4576"[0m [37m+ }[0m [37m+ #---------SVC4577------------------------------[0m [37m+ # %1 - "resource type"[0m [37m+ RESOURCE_CANNOT_CONTAIN_RESOURCE_INSTANCES: {[0m [37m+ code: 409,[0m [37m+ message: "Error : Resource of type %1 cannot contain resource instances.",[0m [37m+ messageId: "SVC4577"[0m [37m+ }[0m [37m+ #---------SVC4578------------------------------[0m [37m+ # %1 - "Resource"/"Service"[0m [37m+ # %2 - resource/service name[0m [37m+ # %3 - "artifact name"[0m [37m+ DEPLOYMENT_ARTIFACT_NAME_ALREADY_EXISTS: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 '%2' already has a deployment artifact named '%3'.",[0m [37m+ messageId: "SVC4578"[0m [37m+ }[0m [37m+ #---------SVC4579------------------------------[0m [37m+ # %1 - "Category"/"Sub-Category"/"Group"[0m [37m+ # %2 - category/sub-category/grouping name.[0m [37m+ INVALID_GROUP_ASSOCIATION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid group association. %1 '%2' was not found.",[0m [37m+ messageId: "SVC4579"[0m [37m+ }[0m [37m+ #---------SVC4580------------------------------[0m [37m+ EMPTY_PRODUCT_CONTACTS_LIST: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. At least one Product Contact has to be specified.",[0m [37m+ messageId: "SVC4580"[0m [37m+ }[0m [37m+ #---------SVC4581------------------------------[0m [37m+ # %1 - UserId[0m [37m+ INVALID_PRODUCT_CONTACT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. User '%1' cannot be set as Product Contact.",[0m [37m+ messageId: "SVC4581"[0m [37m+ }[0m [37m+ #---------SVC4582------------------------------[0m [37m+ # %1 - Product[0m [37m+ # %2 - "abbreviated"/"full"[0m [37m+ MISSING_ONE_OF_COMPONENT_NAMES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. Missing %1 %2 name.",[0m [37m+ messageId: "SVC4582"[0m [37m+ }[0m [37m+ #---------SVC4583------------------------------[0m [37m+ # %1 - "Icon"[0m [37m+ # %2 - "resource"/"service"/"product"[0m [37m+ COMPONENT_PARAMETER_CANNOT_BE_CHANGED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 cannot be changed once the %2 is certified.",[0m [37m+ messageId: "SVC4583"[0m [37m+ }[0m [37m+ #---------SVC4584------------------------------[0m [37m+ # %1 - service/VF name[0m [37m+ # %2 - "service" /"VF"[0m [37m+ # %3 - resource instance origin type[0m [37m+ # %4 - resource instance name[0m [37m+ # %5 - requirement/capability[0m [37m+ # %6 - requirement/capability name[0m [37m+ # %7 - "fulfilled" (for req)/"consumed (for cap)"[0m [37m+ REQ_CAP_NOT_SATISFIED_BEFORE_CERTIFICATION: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Requested '%1' %2 is not ready for certification. %3 '%4' has to have %5 '%6' %7.",[0m [37m+ messageId: "SVC4584"[0m [37m+ }[0m [37m+ #---------SVC4585------------------------------[0m [37m+ INVALID_OCCURRENCES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid occurrences format.",[0m [37m+ messageId: "SVC4585"[0m [37m+ }[0m [37m+ #---------SVC4586------------------------------[0m [37m+ #---------SVC4586------------------------------[0m [37m+ INVALID_SERVICE_API_URL: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Service API URL. Please check whether your URL has a valid domain extension and does not contain the following characters - #?&@%+;,=$<>~^`\[]{}|"*!',[0m [37m+ messageId: "SVC4586"[0m [37m+ }[0m [37m+ #---------SVC4587------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ DATA_TYPE_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: 'Error: Data type %1 already exists.',[0m [37m+ messageId: "SVC4587"[0m [37m+ }[0m [37m+ #---------SVC4588------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ DATA_TYPE_NOR_PROPERTIES_NEITHER_DERIVED_FROM: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Data type %1. Data type must have either a valid derived from declaration or at least one valid property',[0m [37m+ messageId: "SVC4588"[0m [37m+ }[0m [37m+ #---------SVC4589------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ DATA_TYPE_PROPERTIES_CANNOT_BE_EMPTY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Data type %1. 'properties' parameter cannot be empty if provided.",[0m [37m+ messageId: "SVC4589"[0m [37m+ }[0m [37m+ #---------SVC4590------------------------------[0m [37m+ # %1 - Property type name[0m [37m+ # %2 - Property name[0m [37m+ INVALID_PROPERTY_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Property type %1 in property %2.",[0m [37m+ messageId: "SVC4590"[0m [37m+ }[0m [37m+ #---------SVC4591------------------------------[0m [37m+ # %1 - Property inner type[0m [37m+ # %2 - Property name[0m [37m+ INVALID_PROPERTY_INNER_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid property inner type %1, in property %2",[0m [37m+ messageId: "SVC4591"[0m [37m+ }[0m [37m+ #---------SVC4592------------------------------[0m [37m+ # %1 - component instance name[0m [37m+ # %2 - "resource instance"/"service instance"[0m [37m+ COMPONENT_INSTANCE_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' %2 was not found.",[0m [37m+ messageId: "SVC4592"[0m [37m+ }[0m [37m+ #---------SVC4593------------------------------[0m [37m+ # %1 - component instance name[0m [37m+ # %2 - "resource instance"/"service instance"[0m [37m+ # %3 - "resource/"service"/"product"[0m [37m+ # %4 - container name[0m [37m+ COMPONENT_INSTANCE_NOT_FOUND_ON_CONTAINER: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' %2 was not found on the %3 '%4'.",[0m [37m+ messageId: "SVC4593"[0m [37m+ }[0m [37m+ #---------SVC4594------------------------------[0m [37m+ #%1 - requirement / capability[0m [37m+ #%2 - requirement name[0m [37m+ IMPORT_DUPLICATE_REQ_CAP_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Imported TOSCA template contains more than one %1 named '%2'.",[0m [37m+ messageId: "SVC4594"[0m [37m+ }[0m [37m+ #---------SVC4595------------------------------[0m [37m+ #%1 - requirement / capability[0m [37m+ #%2 - requirement name[0m [37m+ #%3 - parent containing the requirement[0m [37m+ IMPORT_REQ_CAP_NAME_EXISTS_IN_DERIVED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Imported TOSCA template contains %1 '%2' that is already defined by derived template %3.",[0m [37m+ messageId: "SVC4595"[0m [37m+ }[0m [37m+ #---------SVC4596------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ DATA_TYPE_DERIVED_IS_MISSING: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. The ancestor data type %1 cannot be found in the system.",[0m [37m+ messageId: "SVC4596"[0m [37m+ }[0m [37m+ #---------SVC4597------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ # %2 - Property names[0m [37m+ DATA_TYPE_PROPERTY_ALREADY_DEFINED_IN_ANCESTOR: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. The data type %1 contains properties named %2 which are already defined in one of its ancestors.",[0m [37m+ messageId: "SVC4597"[0m [37m+ }[0m [37m+ #---------SVC4598------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ DATA_TYPE_DUPLICATE_PROPERTY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. The data type %1 contains duplicate property.",[0m [37m+ messageId: "SVC4598"[0m [37m+ }[0m [37m+ #---------SVC4599------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ # %2 - Property names[0m [37m+ DATA_TYPE_PROEPRTY_CANNOT_HAVE_SAME_TYPE_OF_DATA_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. The data type %1 contains properties %2 which their type is this data type.",[0m [37m+ messageId: "SVC4599"[0m [37m+ }[0m [37m+ #---------SVC4600------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ DATA_TYPE_CANNOT_HAVE_PROPERTIES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. The data type %1 cannot have properties since it is of type scalar",[0m [37m+ messageId: "SVC4600"[0m [37m+ }[0m [37m+ #---------SVC4601------------------------------[0m [37m+ NOT_TOPOLOGY_TOSCA_TEMPLATE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: TOSCA yaml file %1 cannot be modeled to VF as it does not contain 'topology_template.",[0m [37m+ messageId: "SVC4601"[0m [37m+ }[0m [37m+ #---------SVC4602--------------------------------[0m [37m+ # %1 - yaml file name[0m [37m+ # %2 - node_template label[0m [37m+ # %3 - node_template type[0m [37m+ INVALID_NODE_TEMPLATE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: TOSCA yaml file '%1' contains node_template '%2' of type '%3' that does not represent existing VFC/CP/VL",[0m [37m+ messageId: "SVC4602"[0m [37m+ }[0m [37m+ #---------SVC4603------------------------------[0m [37m+ # %1 - component type[0m [37m+ # %2 - component name[0m [37m+ # %3 - state[0m [37m+ ILLEGAL_COMPONENT_STATE: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Action is not allowed on %1 '%2' because it is in an illegal state %3.",[0m [37m+ messageId: "SVC4603"[0m [37m+ }[0m [37m+ #---------SVC4604------------------------------[0m [37m+ # %1 - csar file name[0m [37m+ CSAR_INVALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: TOSCA CSAR '%1' is invalid. 'TOSCA-Metadata/Tosca.meta' file must be provided.",[0m [37m+ messageId: "SVC4604"[0m [37m+ }[0m [37m+ #---------SVC4605------------------------------[0m [37m+ # %1 - csar file name[0m [37m+ CSAR_INVALID_FORMAT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: TOSCA CSAR '%1' is invalid. Invalid 'TOSCA-Metadata/Tosca.meta' file format.",[0m [37m+ messageId: "SVC4605"[0m [37m+ }[0m [37m+ #---------SVC4606------------------------------[0m [37m+ # %1 - property name[0m [37m+ # %2 - property type[0m [37m+ # %3 - property innerType[0m [37m+ # %4 - default value is[0m [37m+ INVALID_COMPLEX_DEFAULT_VALUE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid default value of property %1. Data type is %2 with inner type %3 and default value found is %4.",[0m [37m+ messageId: "SVC4606"[0m [37m+ }[0m [37m+ #---------SVC4607------------------------------[0m [37m+ # %1 - csar file name[0m [37m+ CSAR_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: TOSCA CSAR '%1' is not found.",[0m [37m+ messageId: "SVC4607"[0m [37m+ }[0m [37m+ #---------SVC4608------------------------------[0m [37m+ # %1 - artifact name[0m [37m+ # %2 - component type[0m [37m+ # %3 - actual component type[0m [37m+ MISMATCH_BETWEEN_ARTIFACT_TYPE_AND_COMPONENT_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Artifact %1 is only compatible with component of type %2, but component type is %3.",[0m [37m+ messageId: "SVC4608"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4609------------------------------[0m [37m+ # %1 - "INVALID_JSON"[0m [37m+ INVALID_JSON: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Uploaded JSON file for %1 artifact is invalid.",[0m [37m+ messageId: "SVC4609"[0m [37m+ }[0m [37m+ #---------SVC4610------------------------------[0m [37m+ # %1 - csar file name[0m [37m+ # %2 - missing file name[0m [37m+ YAML_NOT_FOUND_IN_CSAR: {[0m [37m+ code: 400,[0m [37m+ message: "Error - TOSCA CSAR %1 is invalid. TOSCA-Metadata/Tosca.meta refers to file %2 that is not provided.",[0m [37m+ messageId: "SVC4610"[0m [37m+ }[0m [37m+ #---------SVC4611------------------------------[0m [37m+ # %1 - group name[0m [37m+ GROUP_MEMBER_EMPTY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Group %1 member list was provided but does not have values",[0m [37m+ messageId: "SVC4611"[0m [37m+ }[0m [37m+ #---------SVC4612------------------------------[0m [37m+ # %1 - group name[0m [37m+ GROUP_TYPE_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: 'Error: Group type %1 already exists.',[0m [37m+ messageId: "SVC4612"[0m [37m+ }[0m [37m+ #---------SVC4613------------------------------[0m [37m+ # %1 - group name[0m [37m+ # %2 - VF name(component name)[0m [37m+ # %3 - actual component type [VF][0m [37m+ GROUP_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Group with name '%1' already exists in %2 %3.",[0m [37m+ messageId: "SVC4613"[0m [37m+ }[0m [37m+ #---------SVC4614------------------------------[0m [37m+ # %1 - group type[0m [37m+ GROUP_TYPE_IS_INVALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. Group type %1 does not exist",[0m [37m+ messageId: "SVC4614"[0m [37m+ }[0m [37m+ #---------SVC4615------------------------------[0m [37m+ # %1 - group name[0m [37m+ GROUP_MISSING_GROUP_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Missing Group Type for group '%1'",[0m [37m+ messageId: "SVC4615"[0m [37m+ }[0m [37m+ #---------SVC4616------------------------------[0m [37m+ # %1 - member name[0m [37m+ # %2 - group name[0m [37m+ # %3 - VF name[0m [37m+ # %4 - component type [VF ][0m [37m+ GROUP_INVALID_COMPONENT_INSTANCE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Member '%1' listed in group '%2' is not part of '%3' %4.",[0m [37m+ messageId: "SVC4616"[0m [37m+ }[0m [37m+ #---------SVC4617------------------------------[0m [37m+ # %1 - member name[0m [37m+ # %2 - group name[0m [37m+ # %3 - group type[0m [37m+ GROUP_INVALID_TOSCA_NAME_OF_COMPONENT_INSTANCE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: member %1 listed in group %2 is not part of allowed members of group type %3.",[0m [37m+ messageId: "SVC4617"[0m [37m+ }[0m [37m+ #---------SVC4618------------------------------[0m [37m+ # %1 - missing file name[0m [37m+ # %2 - csar file name[0m [37m+ ARTIFACT_NOT_FOUND_IN_CSAR: {[0m [37m+ code: 400,[0m [37m+ message: "Error: artifact %1 is defined in CSAR %2 manifest but is not provided",[0m [37m+ messageId: "SVC4618"[0m [37m+ }[0m [37m+#---------SVC4619------------------------------[0m [37m+# %1 - artifact name[0m [37m+# %2 - artifact type[0m [37m+# %3 - existing artifact type[0m [37m+ ARTIFACT_ALREADY_EXIST_IN_DIFFERENT_TYPE_IN_CSAR: {[0m [37m+ code: 400,[0m [37m+ message: "Error: artifact %1 in type %2 already exists in type %3.",[0m [37m+ messageId: "SVC4619"[0m [37m+ }[0m [37m+ #---------SVC4620------------------------------[0m [37m+ FAILED_RETRIVE_ARTIFACTS_TYPES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Failed to retrieve list of suported artifact types.",[0m [37m+ messageId: "SVC4620"[0m [37m+ }[0m [37m+ #---------SVC4621------------------------------[0m [37m+ # %1 - artifact name[0m [37m+ # %2 - master[0m [37m+ ARTIFACT_ALRADY_EXIST_IN_MASTER_IN_CSAR: {[0m [37m+ code: 400,[0m [37m+ message: "Error: artifact %1 already exists in master %2 .",[0m [37m+ messageId: "SVC4621"[0m [37m+ }[0m [37m+ #---------SVC4622------------------------------[0m [37m+ # %1 - artifact name[0m [37m+ # %2 - artifact type[0m [37m+ # %3 - master name[0m [37m+ # %4 - master type[0m [37m+ ARTIFACT_NOT_VALID_IN_MASTER: {[0m [37m+ code: 400,[0m [37m+ message: "Error: artifact %1 in type %2 can not be exists under master %3 in type %4.",[0m [37m+ messageId: "SVC4622"[0m [37m+ }[0m [37m+ #---------SVC4623------------------------------[0m [37m+ # %1 - artifact name[0m [37m+ # %2 - artifact type[0m [37m+ # %3 - env name[0m [37m+ # %4 - existing env[0m [37m+ ARTIFACT_NOT_VALID_ENV: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Artifact %1 in type %2 with env %3 already exists with another env %4",[0m [37m+ messageId: "SVC4623"[0m [37m+ }[0m [37m+ #---------SVC4624------------------------------[0m [37m+ # %1 - groups names[0m [37m+ # %2 - VF name[0m [37m+ # %3 - component type [VF ][0m [37m+ GROUP_IS_MISSING: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. The groups '%1' cannot be found under %2 %3.",[0m [37m+ messageId: "SVC4624"[0m [37m+ }[0m [37m+ #---------SVC4625------------------------------[0m [37m+ # %1 - groups name[0m [37m+ GROUP_ARTIFACT_ALREADY_ASSOCIATED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Artifact already associated to group '%1'.",[0m [37m+ messageId: "SVC4625"[0m [37m+ }[0m [37m+ #---------SVC4626------------------------------[0m [37m+ # %1 - groups name[0m [37m+ GROUP_ARTIFACT_ALREADY_DISSOCIATED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Artifact already dissociated from group '%1'.",[0m [37m+ messageId: "SVC4626"[0m [37m+ }[0m [37m+ #---------SVC4627------------------------------[0m [37m+ # %1 - property name[0m [37m+ # %2 - group name[0m [37m+ # %3 - group type name[0m [37m+ GROUP_PROPERTY_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: property %1 listed in group %2 is not exist in group type %3.",[0m [37m+ messageId: "SVC4627"[0m [37m+ }[0m [37m+ #---------SVC4628------------------------------[0m [37m+ # %1 - csarUUID[0m [37m+ # %2 - VF name[0m [37m+ VSP_ALREADY_EXISTS: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The VSP with UUID %1 was already imported for VF %2. Please select another or update the existing VF.",[0m [37m+ messageId: "SVC4628"[0m [37m+ }[0m [37m+ #---------SVC4629------------------------------[0m [37m+ # %1 - VF name[0m [37m+ MISSING_CSAR_UUID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The Csar UUID or payload name is missing for VF %1.",[0m [37m+ messageId: "SVC4629"[0m [37m+ }[0m [37m+ #---------SVC4630------------------------------[0m [37m+ # %1 - VF name[0m [37m+ # %2 - new csarUUID[0m [37m+ # %3 - old csarUUID[0m [37m+ RESOURCE_LINKED_TO_DIFFERENT_VSP: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Resource %1 cannot be updated using CsarUUID %2 since the resource is linked to a different VSP with csarUUID %3.",[0m [37m+ messageId: "SVC4630"[0m [37m+ }[0m [37m+ #---------SVC4631------------------------------[0m [37m+ # %1 - policy name[0m [37m+ POLICY_TYPE_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Policy type %1 already exists.",[0m [37m+ messageId: "SVC4631"[0m [37m+ }[0m [37m+ #---------SVC4632------------------------------[0m [37m+ # %1 - target name[0m [37m+ # %2 - policy type name[0m [37m+ TARGETS_NON_VALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: target %1 listed in policy type %2 is not a group or resource.",[0m [37m+ messageId: "SVC4632"[0m [37m+ }[0m [37m+ #---------SVC4633------------------------------[0m [37m+ # %1 - policy name[0m [37m+ TARGETS_EMPTY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Policy %1 target list was provided but does not have values",[0m [37m+ messageId: "SVC4633"[0m [37m+ }[0m [37m+ #---------SVC4634------------------------------[0m [37m+ DATA_TYPE_CANNOT_BE_EMPTY: {[0m [37m+ code: 500,[0m [37m+ message: "Error: Data types are empty. Please import the data types.",[0m [37m+ messageId: "SVC4634"[0m [37m+ }[0m [37m+ #---------SVC4635------------------------------[0m [37m+ # %1 - csar uuid[0m [37m+ RESOURCE_FROM_CSAR_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: resource from csar uuid %1 not found",[0m [37m+ messageId: "SVC4635"[0m [37m+ }[0m [37m+ #---------SVC4636------------------------------[0m [37m+ # %1 - Data type name[0m [37m+ DATA_TYPE_CANNOT_BE_UPDATED_BAD_REQUEST: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Data type %1 cannot be upgraded. The new data type does not contain old properties or the type of one of the properties has been changed.',[0m [37m+ messageId: "SVC4636"[0m [37m+ }[0m [37m+ #-----------SVC4637---------------------------[0m [37m+ #%1 - attribute name[0m [37m+ ATTRIBUTE_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' attribute was not found.",[0m [37m+ messageId: "SVC4637"[0m [37m+ }[0m [37m+ #-----------SVC4638---------------------------[0m [37m+ #%1 - attribute name[0m [37m+ ATTRIBUTE_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Attribute with '%1' name already exists.",[0m [37m+ messageId: "SVC4638"[0m [37m+ }[0m [37m+ #-----------SVC4639---------------------------[0m [37m+ #%1 - property name[0m [37m+ PROPERTY_NAME_ALREADY_EXISTS: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Property with '%1' name and different type already exists.",[0m [37m+ messageId: "SVC4639"[0m [37m+ }[0m [37m+ #-----------SVC4640---------------------------[0m [37m+ #%1 - property name[0m [37m+ INVALID_PROPERTY: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Invalid property received.",[0m [37m+ messageId: "SVC4640"[0m [37m+ }[0m [37m+ #---------SVC4641-----------------------------[0m [37m+ #%1 - invalid filter[0m [37m+ #%2 - valid filters[0m [37m+ INVALID_FILTER_KEY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The filter %1 is not applicable. Please use one of the following filters: %2",[0m [37m+ messageId: "SVC4641"[0m [37m+ }[0m [37m+ #---------SVC4642-----------------------------[0m [37m+ #%1 - asset type[0m [37m+ #%2 - filter[0m [37m+ NO_ASSETS_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "No %1 were found to match criteria %2",[0m [37m+ messageId: "SVC4642"[0m [37m+ }[0m [37m+ #---------SVC4643------------------------------[0m [37m+ # %1 - "Resource"/"Product"[0m [37m+ # %2 - "sub-category name"[0m [37m+ # %3 - "category name"[0m [37m+ COMPONENT_SUB_CATEGORY_NOT_FOUND_FOR_CATEGORY: {[0m [37m+ code: 404,[0m [37m+ message: "Error: %1 sub-category '%2' not found under category '%3'.",[0m [37m+ messageId: "SVC4643"[0m [37m+ }[0m [37m+ #---------SVC4644------------------------------[0m [37m+ # %1 - Format[0m [37m+ CORRUPTED_FORMAT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 format is corrupted.",[0m [37m+ messageId: "SVC4644"[0m [37m+ }[0m [37m+ #---------SVC4645------------------------------[0m [37m+ # %1 - "groupType"[0m [37m+ INVALID_VF_MODULE_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid group type '%1' (should be VfModule).",[0m [37m+ messageId: "SVC4645"[0m [37m+ }[0m [37m+ #---------SVC4646------------------------------[0m [37m+ # %1 - "groupName"[0m [37m+ INVALID_VF_MODULE_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Group name '%1' contains invalid characters",[0m [37m+ messageId: "SVC4646"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4647------------------------------[0m [37m+ # %1 - "modifiedName"[0m [37m+ INVALID_VF_MODULE_NAME_MODIFICATION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid VF Module name modification, can not modify '%1'",[0m [37m+ messageId: "SVC4647"[0m [37m+ }[0m [37m+ #---------SVC4648------------------------------[0m [37m+ # %1 - "inputId"[0m [37m+ # %2 - "componentId"[0m [37m+ INPUT_IS_NOT_CHILD_OF_COMPONENT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Input id: '%1' is not child of component id: '%2'",[0m [37m+ messageId: "SVC4648"[0m [37m+ }[0m [37m+ #---------SVC4649------------------------------[0m [37m+ # %1 - "groupName"[0m [37m+ GROUP_HAS_CYCLIC_DEPENDENCY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The group '%1' has cyclic dependency",[0m [37m+ messageId: "SVC4649"[0m [37m+ }[0m [37m+ #---------SVC4650------------------------------[0m [37m+ # %1 - "Component Type"[0m [37m+ # %2 -[0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/all.log [0m [37m+[0m [37m+[0m [37m+ [0m [37m+[0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/all.log.%i[0m [37m+ [0m [37m+1 [0m [37m+10 [0m [37m+[0m [37m+ [0m [37m+[0m [37m+20MB [0m [37m+[0m [37m+ [0m [37m+${all-log-pattern} [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/debug.log [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+ENTRY [0m [37m+EXIT [0m [37m+INVOKE [0m [37m+INVOKE-RETURN [0m [37m+INVOKE-SYNCHRONOUS [0m [37m+NEUTRAL [0m [37m+DENY [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+[0m [37m+ e.level.toInt() <= INFO.toInt()[0m [37m+ [0m [37m+DENY [0m [37m+ACCEPT [0m [37m+[0m [37m+ [0m [37m+[0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/debug.log.%i[0m [37m+ [0m [37m+1 [0m [37m+10 [0m [37m+[0m [37m+ [0m [37m+[0m [37m+20MB [0m [37m+[0m [37m+ [0m [37m+${debug-log-pattern} [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/error.log [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+WARN [0m [37m+[0m [37m+ [0m [37m+[0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/error.log.%i[0m [37m+ [0m [37m+1 [0m [37m+10 [0m [37m+[0m [37m+ [0m [37m+[0m [37m+20MB [0m [37m+[0m [37m+ [0m [37m+${error-log-pattern} [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/audit.log [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+ENTRY [0m [37m+EXIT [0m [37m+DENY [0m [37m+ACCEPT [0m [37m+[0m [37m+ [0m [37m+[0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/audit.log.%i[0m [37m+ [0m [37m+1 [0m [37m+10 [0m [37m+[0m [37m+ [0m [37m+[0m [37m+20MB [0m [37m+[0m [37m+ [0m [37m+${audit-log-pattern} [0m [37m+[0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/metrics.log[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+INVOKE [0m [37m+INVOKE-RETURN [0m [37m+INVOKE-SYNCHRONOUS [0m [37m+DENY [0m [37m+ACCEPT [0m [37m+[0m [37m+ [0m [37m+[0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/metrics.log.%i[0m [37m+ [0m [37m+1 [0m [37m+10 [0m [37m+[0m [37m+ [0m [37m+[0m [37m+20MB [0m [37m+[0m [37m+ [0m [37m+${metric-log-pattern} [0m [37m+[0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/supportability.log [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+SUPPORTABILITY_MARKER [0m [37m+DENY [0m [37m+ACCEPT [0m [37m+[0m [37m+ [0m [37m+[0m [37m+${log.home}/${ECOMP-component-name}/${ECOMP-subcomponent-name}/supportability.log.%i[0m [37m+ [0m [37m+1 [0m [37m+10 [0m [37m+[0m [37m+ [0m [37m+[0m [37m+20MB [0m [37m+[0m [37m+ [0m [37m+${supportability-log-pattern} [0m [37m+[0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+ [0m [37m+[0m [37m+[0m [37m+[0m [37m+ [0m [37m+ [0m [37m+[0m [37m+[0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+ [0m [37m+[0m [37m+ [0m [37m+[0m [37m+ [0m [37m+ [0m [37m+ # %3 - error description[0m [37m+ AAI_ARTIFACT_GENERATION_FAILED: {[0m [37m+ code: 500,[0m [37m+ message: "Error: %1 %2 automatic generation of artifacts failed. Description: %3",[0m [37m+ messageId: "SVC4650"[0m [37m+ }[0m [37m+ #---------SVC4651------------------------------[0m [37m+ PARENT_RESOURCE_DOES_NOT_EXTEND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Once resource is certified, derived_from can be changed only to a sibling",[0m [37m+ messageId: "SVC4651"[0m [37m+ }[0m [37m+ #---------SVC4652------------------------------[0m [37m+ # %1 - resource/service[0m [37m+ COMPONENT_INVALID_SUBCATEGORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Invalid %1 sub category.",[0m [37m+ messageId: "SVC4652"[0m [37m+ }[0m [37m+ #---------SVC4653------------------------------[0m [37m+ # %1 - group instance uniqueId[0m [37m+ # %2 - service uniqueId[0m [37m+ GROUP_INSTANCE_NOT_FOUND_ON_COMPONENT_INSTANCE: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested group instance %1 was not found on component %2.",[0m [37m+ messageId: "SVC4653"[0m [37m+ }[0m [37m+ #---------SVC4654------------------------------[0m [37m+ # %1 - group property name[0m [37m+ # %2 - valid min limit value[0m [37m+ # %3 - valid max limit value[0m [37m+ INVALID_GROUP_MIN_MAX_INSTANCES_PROPERTY_VALUE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Value of %1 must be not higher than %2, and not lower than %3.",[0m [37m+ messageId: "SVC4654"[0m [37m+ }[0m [37m+ #---------SVC4655------------------------------[0m [37m+ # %1 - group property name[0m [37m+ # %2 - valid min limit value[0m [37m+ # %3 - valid max limit value[0m [37m+ INVALID_GROUP_INITIAL_COUNT_PROPERTY_VALUE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Value of %1 must be between %2 and %3.",[0m [37m+ messageId: "SVC4655"[0m [37m+ }[0m [37m+ #---------SVC4656------------------------------[0m [37m+ # %1 - group property name[0m [37m+ # %2 - lower/higher[0m [37m+ # %3 - valid max/min value[0m [37m+ INVALID_GROUP_PROPERTY_VALUE_LOWER_HIGHER: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Value of %1 must be %2 or equals to %3.",[0m [37m+ messageId: "SVC4656"[0m [37m+ }[0m [37m+ #---------SVC4657------------------------------[0m [37m+ # %1 - certificationRequest / startTesting[0m [37m+ RESOURCE_VFCMT_LIFECYCLE_STATE_NOT_VALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error - Lifecycle state %1 is not valid for resource of type VFCMT",[0m [37m+ messageId: "SVC4657"[0m [37m+ }[0m [37m+ #---------SVC4658------------------------------[0m [37m+ # %1 – asset type [service / resource ][0m [37m+ # %2 – main asset uuid[0m [37m+ # %3 – not found asset type [service / resource][0m [37m+ # %4 – not found asset name[0m [37m+ ASSET_NOT_FOUND_DURING_CSAR_CREATION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: CSAR packaging failed for %1 %2. %3 %4 was not found",[0m [37m+ messageId: "SVC4658"[0m [37m+ }[0m [37m+ #---------SVC4659------------------------------[0m [37m+ # %1 – asset type [service / resource ][0m [37m+ # %2 – main asset uuid[0m [37m+ # %3 – Artifact name[0m [37m+ # %4 – Artifact uuid[0m [37m+ ARTIFACT_PAYLOAD_NOT_FOUND_DURING_CSAR_CREATION: {[0m [37m+ code: 400,[0m [37m+ message: "Error: CSAR packaging failed for %1 %2. Artifact %3 [%4] was not found",[0m [37m+ messageId: "SVC4659"[0m [37m+ }[0m [37m+ #---------SVC4660------------------------------[0m [37m+ # %1 - assetType[0m [37m+ # %2 - matching generic node type name[0m [37m+ GENERIC_TYPE_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Creation of %1 failed. Generic type %2 was not found",[0m [37m+ messageId: "SVC4660"[0m [37m+ }[0m [37m+ #---------SVC4661------------------------------[0m [37m+ # %1 - assetType[0m [37m+ # %2 - matching generic node type name[0m [37m+ TOSCA_SCHEMA_FILES_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: CSAR packaging failed. TOSCA schema files for SDC-Version: %1 and Conformance-Level %2 were not found",[0m [37m+ messageId: "SVC4661"[0m [37m+ }[0m [37m+ #---------SVC4662------------------------------[0m [37m+ # %1 - file name[0m [37m+ # %2 - parser error[0m [37m+ TOSCA_PARSE_ERROR: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid TOSCA template in file %1. %2",[0m [37m+ messageId: "SVC4662"[0m [37m+ }[0m [37m+ #---------SVC4663------------------------------[0m [37m+ # %1 - max length[0m [37m+ RESOURCE_VENDOR_MODEL_NUMBER_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Resource vendor model number exceeds limit of %1 characters.",[0m [37m+ messageId: "SVC4663"[0m [37m+ }[0m [37m+ #---------SVC4664------------------------------[0m [37m+ INVALID_RESOURCE_VENDOR_MODEL_NUMBER: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. Resource vendor model number is not allowed to contain characters like <>:"\/|?* and space characters other than regular space.',[0m [37m+ messageId: "SVC4664"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4669-----------------------------[0m [37m+ INVALID_RESOURCE_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid resource type.",[0m [37m+ messageId: "SVC4669"[0m [37m+ }[0m [37m+ #---------SVC4670------------------------------[0m [37m+ ARTIFACT_NAME_INVALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Artifact name is invalid.",[0m [37m+ messageId: "SVC4670"[0m [37m+ }[0m [37m+ #---------SVC4671------------------------------[0m [37m+ # %1 - VSP name[0m [37m+ # %2 - VFC name[0m [37m+ CFVC_LOOP_DETECTED: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: VSP %1 cannot be imported. The VSP contains internal loop in VFC %2',[0m [37m+ messageId: "SVC4671"[0m [37m+ }[0m [37m+ #---------SVC4672------------------------------[0m [37m+ # %1 - capability uniqueId[0m [37m+ # %2 - instance uniqueId[0m [37m+ # %3 - container uniqueId[0m [37m+ CAPABILITY_OF_INSTANCE_NOT_FOUND_ON_CONTAINER: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested capability %1 of instance %2 was not found on the container %3.",[0m [37m+ messageId: "SVC4672"[0m [37m+ }[0m [37m+ #---------SVC4673------------------------------[0m [37m+ # %1 - requirement uniqueId[0m [37m+ # %2 - instance uniqueId[0m [37m+ # %3 - container uniqueId[0m [37m+ REQUIREMENT_OF_INSTANCE_NOT_FOUND_ON_CONTAINER: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested requirement %1 of instance %2 was not found on the container %3.",[0m [37m+ messageId: "SVC4673"[0m [37m+ }[0m [37m+ #---------SVC4674-----------------------------[0m [37m+ # %1 - relation Id[0m [37m+ # %2 - container uniqueId[0m [37m+ RELATION_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested relation %1 was not found on the container %2.",[0m [37m+ messageId: "SVC4674"[0m [37m+ }[0m [37m+[0m [37m+[0m [37m+ #---------SVC4675------------------------------[0m [37m+ INVALID_SERVICE_STATE: {[0m [37m+ code: 409,[0m [37m+ message: "Service state is invalid for this action",[0m [37m+ messageId: "SVC4675"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4676------------------------------[0m [37m+ INVALID_RESPONSE_FROM_PROXY: {[0m [37m+ code: 502,[0m [37m+ message: "Error: The server was acting as a gateway or proxy and received an invalid response from the upstream server",[0m [37m+ messageId: "SVC4676"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4677------------------------------[0m [37m+ API_RESOURCE_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested '%1' was not found.",[0m [37m+ messageId: "SVC4677"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4678------------------------------[0m [37m+ BAD_REQUEST_MISSING_RESOURCE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The required resource name/id is missing in the request",[0m [37m+ messageId: "SVC4678"[0m [37m+ }[0m [37m+ #---------SVC4679------------------------------[0m [37m+ # %1 forwarding path name maximum length[0m [37m+ FORWARDING_PATH_NAME_MAXIMUM_LENGTH: {[0m [37m+ code: 400,[0m [37m+ message: "Forwarding path name too long, , maximum allowed 200 characters : '%1'.",[0m [37m+ messageId: "SVC4679"[0m [37m+ }[0m [37m+ #---------SVC4680------------------------------[0m [37m+ # %1 Forwarding path name already in use[0m [37m+ FORWARDING_PATH_NAME_ALREADY_IN_USE: {[0m [37m+ code: 400,[0m [37m+ message: "Forwarding path name already in use : '%1'.",[0m [37m+ messageId: "SVC4680"[0m [37m+ }[0m [37m+ #---------SVC4681------------------------------[0m [37m+ # %1 Forwarding path name empty[0m [37m+ FORWARDING_PATH_NAME_EMPTY: {[0m [37m+ code: 400,[0m [37m+ message: "Forwarding Path Name can't be empty .",[0m [37m+ messageId: "SVC4681"[0m [37m+ }[0m [37m+ #---------SVC4682------------------------------[0m [37m+ # %1 - resource uniqueId[0m [37m+ # %2 - resource component type[0m [37m+ RESOURCE_CANNOT_CONTAIN_POLICIES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The resource %1 type of %2 cannot contain policies.",[0m [37m+ messageId: "SVC4682"[0m [37m+ }[0m [37m+ #---------SVC4683------------------------------[0m [37m+ # %1 - policy uniqueId[0m [37m+ # %2 - component uniqueId[0m [37m+ POLICY_NOT_FOUND_ON_CONTAINER: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Requested policy %1 was not found on the container %2.",[0m [37m+ messageId: "SVC4683"[0m [37m+ }[0m [37m+ #---------SVC4684------------------------------[0m [37m+ # %1 - policy name[0m [37m+ INVALID_POLICY_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid policy name %1 received.",[0m [37m+ messageId: "SVC4684"[0m [37m+ }[0m [37m+ #---------SVC4801------------------------------[0m [37m+ # %1 - policy type[0m [37m+ POLICY_TYPE_IS_INVALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. Policy type %1 does not exist",[0m [37m+ messageId: "SVC4801"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4802------------------------------[0m [37m+ # %1 - policy name[0m [37m+ POLICY_MISSING_POLICY_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. Type name is not defined for policy %1",[0m [37m+ messageId: "SVC4802"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4685------------------------------[0m [37m+ # %1 - policy name[0m [37m+ POLICY_NAME_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: The policy with the name %1 already exists.",[0m [37m+ messageId: "SVC4685"[0m [37m+ }[0m [37m+ #---------SVC4686------------------------------[0m [37m+ # %1 - policy name[0m [37m+ POLICY_TARGET_DOES_NOT_EXIST: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The targets %1 are not valid, all targets have to be on the topologyTemplate.",[0m [37m+ messageId: "SVC4686"[0m [37m+ }[0m [37m+ #---------SVC4687------------------------------[0m [37m+ # %1 - policy type[0m [37m+ # %2 - component type[0m [37m+ EXCLUDED_POLICY_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The policy of the type %1 excluded to add to a component of the type %2.",[0m [37m+ messageId: "SVC4687"[0m [37m+ }[0m [37m+ #---------SVC4688------------------------------[0m [37m+ # %1 - group type[0m [37m+ # %2 - component type[0m [37m+ GROUP_TYPE_ILLEGAL_PER_COMPONENT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: group type %1 not permitted in component of type %2",[0m [37m+ messageId: "SVC4688"[0m [37m+ }[0m [37m+ #---------SVC4689------------------------------[0m [37m+ # %1 - group type[0m [37m+ # %2 - component type[0m [37m+ POLICY_TARGET_TYPE_DOES_NOT_EXIST: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The target types %1 are not valid.",[0m [37m+ messageId: "SVC4689"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4690------------------------------[0m [37m+ # %1 forwarding path protocol maximum length[0m [37m+ FORWARDING_PATH_PROTOCOL_MAXIMUM_LENGTH: {[0m [37m+ code: 400,[0m [37m+ message: "Forwarding path protocol too long, , maximum allowed 200 characters : '%1'.",[0m [37m+ messageId: "SVC4690"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4691------------------------------[0m [37m+ # %1 forwarding path destination port maximum length[0m [37m+ FORWARDING_PATH_DESTINATION_PORT_MAXIMUM_LENGTH: {[0m [37m+ code: 400,[0m [37m+ message: "Forwarding path destination port too long, , maximum allowed 200 characters : '%1'.",[0m [37m+ messageId: "SVC4691"[0m [37m+ }[0m [37m+[0m [37m+ #---------POL4692------------------------------[0m [37m+ MISSING_OLD_COMPONENT_INSTANCE: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Missing 'componentInstanceId' HTTP param.",[0m [37m+ messageId: "POL4692"[0m [37m+ }[0m [37m+[0m [37m+ #---------POL4693------------------------------[0m [37m+ MISSING_NEW_COMPONENT_INSTANCE: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Missing 'newComponentInstanceId' HTTP param.",[0m [37m+ messageId: "POL4693"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4694------------------------------[0m [37m+ # %1 External Reference Value[0m [37m+ EXT_REF_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: External Reference '%1' was not found.",[0m [37m+ messageId: "SVC4694"[0m [37m+ }[0m [37m+ #---------SVC4695-----------------------------[0m [37m+ # %1 - Interface Operation Name[0m [37m+ INTERFACE_OPERATION_NAME_ALREADY_IN_USE: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Interface Operation name '%1' already in use, Your current changes will not be saved.",[0m [37m+ messageId: "SVC4695"[0m [37m+ }[0m [37m+ #---------SVC4696-----------------------------[0m [37m+ # %1 - Interface Operation Name[0m [37m+ INTERFACE_OPERATION_NAME_INVALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Interface Operation name '%1' is Invalid, Operation name should not contain special character, space and should not be greater than 200 characters.",[0m [37m+ messageId: "SVC4696"[0m [37m+ }[0m [37m+ #---------SVC4697-----------------------------[0m [37m+ INTERFACE_OPERATION_NAME_MANDATORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Interface Operation name is mandatory, Operation name can't be empty.",[0m [37m+ messageId: "SVC4697"[0m [37m+ }[0m [37m+#---------SVC4698-----------------------------[0m [37m+# %1 - Interface type[0m [37m+ INTERFACE_OPERATION_INVALID_FOR_LOCAL_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid input, only one operation is allowed in local interface type '%1'.",[0m [37m+ messageId: "SVC4698"[0m [37m+ }[0m [37m+ #---------SVC4699-----------------------------[0m [37m+ # %1 - Interface Operation input parameter name[0m [37m+ INTERFACE_OPERATION_INPUT_NAME_ALREADY_IN_USE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Interface Operation input parameter name '%1' already in use, Your current changes will not be saved.",[0m [37m+ messageId: "SVC4699"[0m [37m+ }[0m [37m+ #---------SVC4700-----------------------------[0m [37m+ INTERFACE_OPERATION_INPUT_NAME_MANDATORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Interface operation input parameter name should not be empty.",[0m [37m+ messageId: "SVC4700"[0m [37m+ }[0m [37m+#---------SVC4701-----------------------------[0m [37m+# %1 - component Id[0m [37m+ INTERFACE_OPERATION_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Interface operation not found in the component '%1'.",[0m [37m+ messageId: "SVC4701"[0m [37m+ }[0m [37m+ #---------SVC4702-----------------------------[0m [37m+ INTERFACE_OPERATION_NOT_DELETED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Failed to delete interface operation.",[0m [37m+ messageId: "SVC4702"[0m [37m+ }[0m [37m+ #SVC4732[0m [37m+ INTERFACE_UNKNOWN: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The interface '%1' does not exists in the database.",[0m [37m+ messageId: "SVC4732"[0m [37m+ }[0m [37m+ #SVC4733[0m [37m+ INTERFACE_OPERATION_NOT_DEFINED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: The operation '%1' does not exists in the interface '%2'.",[0m [37m+ messageId: "SVC4733"[0m [37m+ }[0m [37m+ #-----------SVC4692---------------------------[0m [37m+ RESOURCE_LIFECYCLE_STATE_NOT_VALID: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Lifecycle state %1 is not valid for resource",[0m [37m+ messageId: "SVC4692"[0m [37m+ }[0m [37m+ #-----------SVC4693---------------------------[0m [37m+ #%1 - component name[0m [37m+ COMPONENT_IS_ARCHIVED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Component %1 is archived",[0m [37m+ messageId: "SVC4693"[0m [37m+ }[0m [37m+ #-----------SVC4703---------------------------[0m [37m+ #%1 - component name[0m [37m+ COMPONENT_IS_NOT_HIHGEST_CERTIFIED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Component %1 is not highest certified",[0m [37m+ messageId: "SVC4703"[0m [37m+ }[0m [37m+ #---------SVC4704------------------------------[0m [37m+ # %1 - "service"/"VF"[0m [37m+ # %2 - "Resource name"[0m [37m+ ARCHIVED_ORIGINS_FOUND: {[0m [37m+ code: 403,[0m [37m+ message: "Error: Action is not permitted as your '%1' '%2' includes archived resources",[0m [37m+ messageId: "SVC4704"[0m [37m+ }[0m [37m+ #---------SVC4705------------------------------[0m [37m+ # %1-artifact name[0m [37m+ ARTIFACT_PAYLOAD_EMPTY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid content. Uploaded file %1 is empty. Please retry with the correct file.",[0m [37m+ messageId: "SVC4705"[0m [37m+ }[0m [37m+ #---------SVC4800------------------------------[0m [37m+ # %1 - "component id"[0m [37m+ UPDATE_CATALOG_FAILED: {[0m [37m+ code: 403,[0m [37m+ message: "Error: update catalog for component '%1' failed.",[0m [37m+ messageId: "SVC4800"[0m [37m+ }[0m [37m+ #---------SVC4706------------------------------[0m [37m+ # %1-input(s) name(s) string[0m [37m+ INPUTS_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: missing input value(s) %1.",[0m [37m+ messageId: "SVC4706"[0m [37m+ }[0m [37m+ #---------SVC4707-----------------------------[0m [37m+ # %1 – asset type [service / resource ][0m [37m+ # %2 – main asset uuid[0m [37m+ ERROR_DURING_CSAR_CREATION: {[0m [37m+ code: 404,[0m [37m+ message: "Error: CSAR packaging failed for %1 %2.",[0m [37m+ messageId: "SVC4706"[0m [37m+ }[0m [37m+#---------SVC4708-----------------------------[0m [37m+# %1 - Interface Operation input property name, component type[0m [37m+ INTERFACE_OPERATION_INPUT_PROPERTY_NOT_FOUND_IN_COMPONENT: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Interface operation input parameter property '%1' not found in '%2' input properties, capability properties or outputs of other operations.",[0m [37m+ messageId: "SVC4708"[0m [37m+ }[0m [37m+#---------SVC4709-----------------------------[0m [37m+# %1 - Interface Operation output parameter name[0m [37m+ INTERFACE_OPERATION_OUTPUT_NAME_ALREADY_IN_USE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Interface Operation output parameter name '%1' already in use, Your current changes will not be saved.",[0m [37m+ messageId: "SVC4708"[0m [37m+ }[0m [37m+#---------SVC4710-----------------------------[0m [37m+ INTERFACE_OPERATION_OUTPUT_NAME_MANDATORY: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Interface operation output parameter name should not be empty.",[0m [37m+ messageId: "SVC4710"[0m [37m+ }[0m [37m+#---------SVC4711-----------------------------[0m [37m+# %1 - interface Id[0m [37m+ INTERFACE_NOT_FOUND_IN_COMPONENT: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Interface not found in the component '%1'.",[0m [37m+ messageId: "SVC4711"[0m [37m+ }[0m [37m+ #---------SVC4709-----------------------------[0m [37m+ INVALID_PROPERTY_CONSTRAINTS: {[0m [37m+ # %1 – property constraints type[0m [37m+ # %2 – received property constraints value[0m [37m+ # %3 – property type[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid %1 %2 for the type %3 have been received.",[0m [37m+ messageId: "SVC4709"[0m [37m+ }[0m [37m+ #---------SVC4710-----------------------------[0m [37m+ INVALID_PROPERTY_CONSTRAINTS_FORMAT: {[0m [37m+ # %1 – received property constraints json section[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid format of the received property constraints section: %1. The property constraints section should be a list.",[0m [37m+ messageId: "SVC4710"[0m [37m+ }[0m [37m+ #---------SVC4711-----------------------------[0m [37m+ CANNOT_DELETE_VALID_VALUES: {[0m [37m+ # %1 – property constraints type[0m [37m+ # %2 – missing valid values[0m [37m+ code: 400,[0m [37m+ message: "Error: Deletion of existing %1 is not permitted on an update. Missing values: %2",[0m [37m+ messageId: "SVC4711"[0m [37m+ }[0m [37m+ #---------SVC4712------------------------------[0m [37m+ MISSING_PROPERTY_NAME: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Invalid Content. Missing mandatory parameter 'name'." ,[0m [37m+ messageId: "SVC4712"[0m [37m+ }[0m [37m+ #---------SVC4713------------------------------[0m [37m+ MISSING_PROPERTY_VALUE: {[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Invalid Content. Missing mandatory parameter 'value'." ,[0m [37m+ messageId: "SVC4713"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4712---------------------------[0m [37m+ INVALID_INSTANTIATION_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Invalid instantiationType: %1",[0m [37m+ messageId: "SVC4712"[0m [37m+ }[0m [37m+[0m [37m+ #----------SVC4713---------------------------[0m [37m+ MISSING_ECOMP_GENERATED_NAMING: {[0m [37m+ code: 400,[0m [37m+ message: "Missing ecompGeneratedNaming property",[0m [37m+ messageId: "SVC4713"[0m [37m+ }[0m [37m+[0m [37m+ #-----------SVC4714--------------------------[0m [37m+ NAMING_POLICY_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. Naming policy exceeds limit of %1 characters.",[0m [37m+ messageId: "SVC4714"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4715------------------------------[0m [37m+ INVALID_NAMING_POLICY: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. Naming policy is not allowed to contain characters like <>:"\/|?* and space characters other than regular space.',[0m [37m+ messageId: "SVC4715"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4716------------------------------[0m [37m+ INVALID_ENVIRONMENT_CONTEXT: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Environment context: %1',[0m [37m+ messageId: "SVC4716"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4717------------------------------[0m [37m+ UNSUPPORTED_DISTRIBUTION_STATUS: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Unsupported distribution action: %1',[0m [37m+ messageId: "SVC4717"[0m [37m+ }[0m [37m+ #---------SVC4718------------------------------[0m [37m+ CONTAINER_CANNOT_CONTAIN_INSTANCE: {[0m [37m+ # %1 - "container type"[0m [37m+ # %2- “component type”[0m [37m+ code: 400 ,[0m [37m+ message: "Error : %1 can’t contain component instance %2" ,[0m [37m+ messageId: "SVC4718"[0m [37m+ }[0m [37m+ #---------SVC4719------------------------------[0m [37m+ CONTAINER_CANNOT_CONTAIN_COMPONENT_IN_STATE: {[0m [37m+ # %1 - "container type"[0m [37m+ # %2- "lifecycle state"[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Container %1 can’t contain component in state %2" ,[0m [37m+ messageId: "SVC4719"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4720------------------------------[0m [37m+ MISSING_MANDATORY_PROPERTY: {[0m [37m+ # %1 - "property name"[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Missing mandatory %1 property" ,[0m [37m+ messageId: "SVC4720"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4721------------------------------[0m [37m+ MANDATORY_PROPERTY_MISSING_VALUE: {[0m [37m+ # %1 - "property name"[0m [37m+ code: 400 ,[0m [37m+ message: "Error: Missing value for the mandatory %1 property" ,[0m [37m+ messageId: "SVC4721"[0m [37m+ }[0m [37m+#---------SVC4712-----------------------------[0m [37m+ INTERFACE_LIFECYCLE_TYPES_NOT_FOUND: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Interface Lifecycle types not found.",[0m [37m+ messageId: "SVC4712"[0m [37m+ }[0m [37m+#---------SVC4713-----------------------------[0m [37m+# %1 - Interface Operation Name[0m [37m+ INTERFACE_OPERATION_INVALID_FOR_GLOBAL_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid input, only pre-defined operation names are allowed in global interface type '%1'",[0m [37m+ messageId: "SVC4713"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4714-----------------------------[0m [37m+ NODE_FILTER_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Node Filter was not found",[0m [37m+ messageId: "SVC4714"[0m [37m+ }[0m [37m+ #---------SVC4715----------------------------[0m [37m+ UNSUPPORTED_VALUE_PROVIDED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Supported value type is %1 for %2 property. Provided Value: %3",[0m [37m+ messageId: "SVC4715"[0m [37m+ }[0m [37m+ #---------SVC4716----------------------------[0m [37m+ # %1 - Property Name[0m [37m+ SELECTED_PROPERTY_NOT_PRESENT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 property does not exists in Service anymore.",[0m [37m+ messageId: "SVC4716"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4717----------------------------[0m [37m+ # %1 - Property Name[0m [37m+ MAPPED_PROPERTY_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 property does not exist.",[0m [37m+ messageId: "SVC4717"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4718----------------------------[0m [37m+ # %1 - Property Name[0m [37m+ # %2 - Operator Name[0m [37m+ UNSUPPORTED_OPERATOR_PROVIDED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 property does not support %2 operator.",[0m [37m+ messageId: "SVC4718"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4719----------------------------[0m [37m+ CONSTRAINT_FORMAT_INCORRECT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Constraint provided does not contains expected values.",[0m [37m+ messageId: "SVC4719"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4720----------------------------[0m [37m+ # %1 - Property Name[0m [37m+ # %2 - Operator Type[0m [37m+ SOURCE_TARGET_PROPERTY_TYPE_MISMATCH: {[0m [37m+ code: 400,[0m [37m+ message: "Error: %1 property and %2 property type is not same.",[0m [37m+ messageId: "SVC4720"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4721----------------------------[0m [37m+ # %1 - Property Type[0m [37m+ # %2 - Operator Type[0m [37m+ UNSUPPORTED_PROPERTY_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Property type %1 provided against %2 is not supported for static value.",[0m [37m+ messageId: "SVC4721"[0m [37m+ }[0m [37m+[0m [37m+ #---------SVC4722------------------------------[0m [37m+ # %1 Directive value set[0m [37m+ DIRECTIVES_INVALID_VALUE: {[0m [37m+ code: 404,[0m [37m+ message: "Error: Invalid directive value : '%1' .",[0m [37m+ messageId: "SVC4722"[0m [37m+ }[0m [37m+#---------SVC4723-----------------------------[0m [37m+# %1 - Interface Operation output name[0m [37m+ INTERFACE_OPERATION_MAPPED_OUTPUT_MODIFIED: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Cannot update or delete interface operation output(s) '%1' mapped to an operation input",[0m [37m+ messageId: "SVC4723"[0m [37m+ }[0m [37m+#---------SVC4724-----------------------------[0m [37m+# %1 - Interface Operation output name[0m [37m+ INTERFACE_OPERATION_DELETE_WITH_MAPPED_OUTPUT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Cannot delete interface operation with output(s) '%1' mapped to another operation input",[0m [37m+ messageId: "SVC4724"[0m [37m+ }[0m [37m+#---------SVC4725-----------------------------[0m [37m+ INVALID_CONSUMPTION_TYPE: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Given value is different than input type. Needs to be %1",[0m [37m+ messageId: "SVC4725"[0m [37m+ }[0m [37m+#---------SVC4726-----------------------------[0m [37m+ INVALID_PROPERTY_VALUES: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid property values provided:\n %1",[0m [37m+ messageId: "SVC4726"[0m [37m+ }[0m [37m+#---------SVC4727------------------------------[0m [37m+ INVALID_PROPERTY_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Property name contains invalid characters. It should have only letters, numbers and underscores.",[0m [37m+ messageId: "SVC4727"[0m [37m+ }[0m [37m+[0m [37m+#---------SVC4728------------------------------[0m [37m+ FAILED_TO_CREATE_OR_UPDATE_CAPABILITY_PROPERTIES: {[0m [37m+ code: 500,[0m [37m+ message: "Error: Failed to create or update capabilities properties",[0m [37m+ messageId: "SVC4728"[0m [37m+ }[0m [37m+[0m [37m+#---------SVC4729------------------------------[0m [37m+ # %1 - resource Id[0m [37m+ CAPABILITY_PROPERTIES_NOT_FOUND: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Capability properties not found in the resource '%1'.",[0m [37m+ messageId: "SVC4729"[0m [37m+ }[0m [37m+#---------SVC4730------------------------------[0m [37m+ # %1 - property name[0m [37m+ PROPERTY_EXCEEDS_LIMIT: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Invalid Content. %1 exceeds limit.",[0m [37m+ messageId: "SVC4722"[0m [37m+ }[0m [37m+#---------SVC4731------------------------------[0m [37m+ INVALID_PROPERY: {[0m [37m+ # %1 - property name[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid Content. %1 has invalid format.',[0m [37m+ messageId: "SVC4723"[0m [37m+ }[0m [37m+#---------SVC4734------------------------------[0m [37m+ # %1 - list of validation errors[0m [37m+ INVALID_PM_DICTIONARY_FILE: {[0m [37m+ code: 400,[0m [37m+ message: 'Error: Invalid PM Dictionary File. %1',[0m [37m+ messageId: "SVC4734"[0m [37m+ }[0m [37m+#-----------SVC4735---------------------------[0m [37m+ #%1 - input name[0m [37m+ INPUT_ALREADY_EXIST: {[0m [37m+ code: 409,[0m [37m+ message: "Error: Input with '%1' name already exists.",[0m [37m+ messageId: "SVC4735"[0m [37m+ }[0m [37m+#---------SVC4736------------------------------[0m [37m+ INVALID_INPUT_NAME: {[0m [37m+ code: 400,[0m [37m+ message: "Error: Input name contains invalid characters. It should have only letters, numbers and underscores.",[0m [37m+ messageId: "SVC4736"[0m [37m+ }[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] mode changed to 644 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/config/catalog-be/error-configuration.yaml] mode changed to 644 [32m- change mode from '' to '0644'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0mRecipe: sdc-catalog-be::BE_9_prepareProbeFile[0m * template[/var/lib/jetty/ready-probe.sh] action create[2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/ready-probe.sh] created file /var/lib/jetty/ready-probe.sh [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/ready-probe.sh] created file /var/lib/jetty/ready-probe.sh [32m- create new file /var/lib/jetty/ready-probe.sh[0m[2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/ready-probe.sh] updated file contents /var/lib/jetty/ready-probe.sh [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/ready-probe.sh] updated file contents /var/lib/jetty/ready-probe.sh [32m- update content in file /var/lib/jetty/ready-probe.sh from none to 82c4e1[0m [32m- suppressed sensitive resource[0m[2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/ready-probe.sh] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/ready-probe.sh] mode changed to 755 [32m- change mode from '' to '0755'[0m [0mRecipe: sdc-catalog-be::BE_10_setup_cadi[0m * directory[Jetty_etc_dir_creation] action create (skipped due to not_if) * cookbook_file[/var/lib/jetty/etc/keyfile] action create[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] created file /var/lib/jetty/etc/keyfile [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] created file /var/lib/jetty/etc/keyfile [32m- create new file /var/lib/jetty/etc/keyfile[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] updated file contents /var/lib/jetty/etc/keyfile [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] updated file contents /var/lib/jetty/etc/keyfile [32m- update content in file /var/lib/jetty/etc/keyfile from none to 699bb0[0m [37m--- /var/lib/jetty/etc/keyfile 2021-05-28 00:05:52.696761798 +0000[0m [37m+++ /var/lib/jetty/etc/.chef-keyfile20210528-22-cybfmr 2021-05-28 00:05:52.696761798 +0000[0m [37m@@ -1,4 +1,8 @@[0m [37m+#############################################################[0m [37m+# This file should be replaced with jetty keyfile: #[0m [37m+# /var/lib/jetty/base/be/etc/keyfile #[0m [37m+#############################################################[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/keyfile] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] action create[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] created file /var/lib/jetty/etc/cadi_truststore.jks [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] created file /var/lib/jetty/etc/cadi_truststore.jks [32m- create new file /var/lib/jetty/etc/cadi_truststore.jks[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] updated file contents /var/lib/jetty/etc/cadi_truststore.jks [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] updated file contents /var/lib/jetty/etc/cadi_truststore.jks [32m- update content in file /var/lib/jetty/etc/cadi_truststore.jks from none to 9eae24[0m [37m--- /var/lib/jetty/etc/cadi_truststore.jks 2021-05-28 00:05:52.704762321 +0000[0m [37m+++ /var/lib/jetty/etc/.chef-cadi_truststore20210528-22-3t8s7f.jks 2021-05-28 00:05:52.704762321 +0000[0m [37m@@ -1,4 +1,8 @@[0m [37m+##################################################################[0m [37m+## This file should be replaced with jetty cadi_truststore.jks: #[0m [37m+## /var/lib/jetty/base/be/etc/cadi_truststore.jks #[0m [37m+##################################################################[0m[2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: cookbook_file[/var/lib/jetty/etc/cadi_truststore.jks] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m * template[/var/lib/jetty/etc/cadi.properties] action create[2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] created file /var/lib/jetty/etc/cadi.properties [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] created file /var/lib/jetty/etc/cadi.properties [32m- create new file /var/lib/jetty/etc/cadi.properties[0m[2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] updated file contents /var/lib/jetty/etc/cadi.properties [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] updated file contents /var/lib/jetty/etc/cadi.properties [32m- update content in file /var/lib/jetty/etc/cadi.properties from none to ac5e7b[0m [37m--- /var/lib/jetty/etc/cadi.properties 2021-05-28 00:05:52.712762843 +0000[0m [37m+++ /var/lib/jetty/etc/.chef-cadi20210528-22-1l2ahks.properties 2021-05-28 00:05:52.712762843 +0000[0m [37m@@ -1,54 +1,108 @@[0m [37m+# Configure AAF[0m [37m+aaf_locate_url=[0m [37m+[0m [37m+aaf_url=[0m [37m+[0m [37m+#if you are running aaf service from a docker image you have to use aaf service IP and port number[0m [37m+aaf_id=user[0m [37m+#Encrypt the password using AAF Jar[0m [37m+aaf_password=enc:AafEncriptedPassword[0m [37m+# Sample CADI Properties, from CADI 1.4.2[0m [37m+hostname=sdc-be.onap[0m [37m+csp_domain=PROD[0m [37m+[0m [37m+# Add Absolute path to Keyfile[0m [37m+cadi_keyfile=/var/lib/jetty/etc/org.onap.sdc.p12[0m [37m+[0m [37m+[0m [37m+# This is required to accept Certificate Authentication from Certman certificates.[0m [37m+# can be TEST, IST or PROD[0m [37m+aaf_env=TEST[0m [37m+[0m [37m+# DEBUG prints off all the properties. Use to get started.[0m [37m+cadi_loglevel=DEBUG[0m [37m+[0m [37m+[0m [37m+# Become CSO Poodle Compliant by only allowing sanctioned TLS versions[0m [37m+# The following is the default[0m [37m+# cadi_protocols=TLSv1.1,TLSv1.2[0m [37m+[0m [37m+# Default TrustStore - REQUIRED for changing PROTOCOL Defaults for DME2[0m [37m+# Read https://wiki.web.att.com/pages/viewpage.action?pageId=574623569#URGENT:SolvingSSL2-3/TLSv1removalissues-Up-to-dateTruststore[0m [37m+# Add Absolute path to truststore2020.jks[0m [37m+cadi_truststore=/var/lib/jetty/etc/org.onap.sdc.trust.jks[0m [37m+# Note: This is the ONLY password that doesn't have to be encrypted. All Java's TrustStores are this passcode by default, because they are public certs[0m [37m+cadi_truststore_password=changeit[0m [37m+[0m [37m+# how to turn on SSL Logging[0m [37m+#javax.net.debug=ssl[0m [37m+[0m [37m+##[0m [37m+# Hint[0m [37m+# Use "maps.bing.com" to get Lat and Long for an Address[0m [37m+AFT_LATITUDE=32.780140[0m [37m+AFT_LONGITUDE=-96.800451[0m [37m+AFT_ENVIRONMENT=AFTUAT[0m [37m+AFT_DME2_CLIENT_IGNORE_SSL_CONFIG=true[0m [37m+DME2.DEBUG=true[0m [37m+AFT_DME2_HTTP_EXCHANGE_TRACE_ON=true[0m [37m+[0m [37m+cadi_latitude=32.780140[0m [37m+cadi_longitude=-96.800451[0m [37m+[0m [37m+aaf_root_ns=com.onap.sdc[0m [37m+aaf_api_version=2.0[0m[2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] owner changed to 352070 [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] group changed to 35953 [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] mode changed to 755 [2021-05-28T00:05:52+00:00] INFO: template[/var/lib/jetty/etc/cadi.properties] mode changed to 755 [32m- change mode from '' to '0755'[0m [32m- change owner from '' to 'jetty'[0m [32m- change group from '' to 'jetty'[0m [0m[2021-05-28T00:05:52+00:00] INFO: Chef Run complete in 1.768929644 seconds [2021-05-28T00:05:52+00:00] INFO: Chef Run complete in 1.768929644 seconds [0m Running handlers:[0m [2021-05-28T00:05:52+00:00] INFO: Running report handlers [2021-05-28T00:05:52+00:00] INFO: Running report handlers Running handlers complete [0m[2021-05-28T00:05:52+00:00] INFO: Report handlers complete [2021-05-28T00:05:52+00:00] INFO: Report handlers complete Chef Client finished, 22/23 resources updated in 03 seconds[0m [2021-05-28T00:05:52+00:00] WARN: ***************************************** [2021-05-28T00:05:52+00:00] WARN: ***************************************** [2021-05-28T00:05:52+00:00] WARN: Did not find config file: /solo.rb, using command line options. [2021-05-28T00:05:52+00:00] WARN: Did not find config file: /solo.rb, using command line options. [2021-05-28T00:05:52+00:00] WARN: ***************************************** [2021-05-28T00:05:52+00:00] WARN: ***************************************** 2021-05-28 00:05:53.393:INFO::main: Logging initialized @582ms to org.eclipse.jetty.util.log.StdErrLog 2021-05-28 00:05:53.718:INFO::main: Console stderr/stdout captured to /var/lib/jetty/logs/2021_05_28.jetty.log
2021-05-28T00:00:07.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:07.273Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:00:10.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:10.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:00:10.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:00:12.291Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:00:12.292Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:00:12.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:00:13.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:16.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:19.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:19.604Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:19.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:00:19.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9d932dc 2021-05-28T00:00:19.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:00:19.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:00:22.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:22.611Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:00:22.611Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:00:22.611Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:00:22.612Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=99e0b781-c925-41c3-87e9-0e4ae73adb85 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:00:22.660Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to /10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:22.677Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:23.664Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:23.665Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:23.675Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:23.675Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:25.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:25.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:00:25.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:00:25.669Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:25.670Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:25.679Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:25.680Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:27.256Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:00:28.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:29.678Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:29.682Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 8000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:29.769Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:29.770Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 8000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:31.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:34.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:37.272Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:00:37.687Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:37.688Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:37.774Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:37.774Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:39.469Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:00:40.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:00:40.518Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:00:42.704Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:42.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a05ed47f-c49d-4437-99b8-f5af3b2178e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:00:42.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a05ed47f-c49d-4437-99b8-f5af3b2178e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@398a3677 2021-05-28T00:00:42.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a05ed47f-c49d-4437-99b8-f5af3b2178e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:00:42.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a05ed47f-c49d-4437-99b8-f5af3b2178e0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:00:42.963Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a05ed47f-c49d-4437-99b8-f5af3b2178e0 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:00:42.964Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=a05ed47f-c49d-4437-99b8-f5af3b2178e0 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:00:43.768Z [cluster2-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:44.905Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 1000 milliseconds 2021-05-28T00:00:53.758Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:53.759Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:53.778Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:00:53.779Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:00:55.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:00:55.518Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:01:03.005Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:01:03.006Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:01:03.006Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24d235fa 2021-05-28T00:01:03.006Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:01:03.006Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:01:03.009Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:01:03.010Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:01:03.010Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:01:03.010Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=741ba767-22ff-45c6-8191-81ac72147ad2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:01:06.005Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 2000 milliseconds 2021-05-28T00:01:10.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:01:10.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:01:12.291Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:01:12.292Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:01:12.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:01:13.776Z [cluster2-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:01:23.105Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:01:23.105Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2829ca-a946-4e23-bc6a-8ce5e76d13e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:01:23.105Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2829ca-a946-4e23-bc6a-8ce5e76d13e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c8c071c 2021-05-28T00:01:23.105Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2829ca-a946-4e23-bc6a-8ce5e76d13e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:01:23.105Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2829ca-a946-4e23-bc6a-8ce5e76d13e0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:01:23.110Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4b2829ca-a946-4e23-bc6a-8ce5e76d13e0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:01:23.110Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4b2829ca-a946-4e23-bc6a-8ce5e76d13e0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:01:25.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:01:25.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:01:25.765Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:01:25.765Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 64000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:01:25.784Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:01:25.785Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 64000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:01:27.280Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:01:28.105Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 4000 milliseconds 2021-05-28T00:01:40.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:01:40.518Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:01:43.204Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:01:43.205Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced3b984-8f7d-4541-9629-18c16fa4c083 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:01:43.205Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced3b984-8f7d-4541-9629-18c16fa4c083 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7358763f 2021-05-28T00:01:43.205Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced3b984-8f7d-4541-9629-18c16fa4c083 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:01:43.205Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced3b984-8f7d-4541-9629-18c16fa4c083 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:01:43.785Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:01:46.215Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ced3b984-8f7d-4541-9629-18c16fa4c083 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:01:46.215Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ced3b984-8f7d-4541-9629-18c16fa4c083 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:01:52.205Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 8000 milliseconds 2021-05-28T00:01:55.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:01:55.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:02:06.304Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:02:06.304Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87da11bd-e9ba-4657-9c93-51afedfe99c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:06.304Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87da11bd-e9ba-4657-9c93-51afedfe99c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16374640 2021-05-28T00:02:06.304Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87da11bd-e9ba-4657-9c93-51afedfe99c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:06.304Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87da11bd-e9ba-4657-9c93-51afedfe99c0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:02:08.480Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e9e5979f-c707-4934-adc9-6b4627ba7750 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:08.479349Z InvocationID=5cf63677-45cd-42d0-b1a1-9636f8492d64 LogTimestamp=2021-05-28T00:02:08.479520Z Entering 2021-05-28T00:02:08.480Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e9e5979f-c707-4934-adc9-6b4627ba7750 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:08.479349Z InvocationID=5cf63677-45cd-42d0-b1a1-9636f8492d64 LogTimestamp=2021-05-28T00:02:08.479520Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:02:08.573Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=93 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:02:08.480Z InvocationID=f011740b-bf27-4abe-a52e-b781eaf72a98 LogTimestamp=2021-05-28T00:02:08.573375Z ResponseDesc=OK Exiting. 2021-05-28T00:02:08.574Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:02:08.684Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=301994 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-27T23:57:06.689Z InvocationID=edd3c1ec-bc3d-4259-bcd7-447b67d2f73c ErrorCode=500 LogTimestamp=2021-05-28T00:02:08.683979Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:08.684Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:02:08.777Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5ff63ec0-2bff-4cb9-8001-62ef8cc20224 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:08.777520Z InvocationID=385ed3f3-e5bc-497d-88b1-813a50e415d8 LogTimestamp=2021-05-28T00:02:08.777563Z Entering 2021-05-28T00:02:08.778Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5ff63ec0-2bff-4cb9-8001-62ef8cc20224 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:08.777520Z InvocationID=385ed3f3-e5bc-497d-88b1-813a50e415d8 LogTimestamp=2021-05-28T00:02:08.777563Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:02:08.981Z [qtp580718781-107] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:02:08.778Z InvocationID=2a1da9bf-4e17-4c6b-8d66-352fb0eb3031 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@28124d17', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 201ms 2021-05-28T00:02:08.983Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=204 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:02:08.778Z InvocationID=2a1da9bf-4e17-4c6b-8d66-352fb0eb3031 ErrorCode=409 LogTimestamp=2021-05-28T00:02:08.982953Z ResponseDesc=Conflict Exiting. 2021-05-28T00:02:08.983Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:02:09.074Z [qtp580718781-108] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.074586Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:09.074Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.074586Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:09.075Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:02:09.083Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fec53218-b592-4602-bb94-7b5ec66814f7 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.083526Z InvocationID=4d3b610f-da23-478d-af8b-3081370641ce LogTimestamp=2021-05-28T00:02:09.083570Z Entering 2021-05-28T00:02:09.083Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fec53218-b592-4602-bb94-7b5ec66814f7 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.083526Z InvocationID=4d3b610f-da23-478d-af8b-3081370641ce LogTimestamp=2021-05-28T00:02:09.083570Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:02:09.273Z [qtp580718781-15] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:02:09.084Z InvocationID=ccee9bba-dd1c-460e-8deb-5f584efa15a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@28124d17', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 188ms 2021-05-28T00:02:09.274Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=190 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:02:09.084Z InvocationID=ccee9bba-dd1c-460e-8deb-5f584efa15a1 ErrorCode=400 LogTimestamp=2021-05-28T00:02:09.274190Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:02:09.274Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:02:09.371Z [qtp580718781-106] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.371740Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:09.371Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.371740Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:09.372Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:02:09.389Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c3a96603-ca9d-44c6-b241-5841673be037 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.388818Z InvocationID=fb02f63f-8021-4af4-bc5c-15f417e152c7 LogTimestamp=2021-05-28T00:02:09.388872Z Entering 2021-05-28T00:02:09.389Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c3a96603-ca9d-44c6-b241-5841673be037 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.388818Z InvocationID=fb02f63f-8021-4af4-bc5c-15f417e152c7 LogTimestamp=2021-05-28T00:02:09.388872Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:02:09.477Z [qtp580718781-14] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:02:09.389Z InvocationID=d0c89307-3173-4df0-8099-e1391d6cb119 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@28124d17', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 87ms 2021-05-28T00:02:09.480Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=89 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:02:09.389Z InvocationID=d0c89307-3173-4df0-8099-e1391d6cb119 ErrorCode=400 LogTimestamp=2021-05-28T00:02:09.478511Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:02:09.480Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:02:09.591Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.591360Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:09.591Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.591360Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:09.592Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:02:09.679Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=21127c5f-bab4-4ff4-9daa-1725e8abf87a InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.679477Z InvocationID=128abe25-56d4-490e-bd71-bdc23d91e5ae LogTimestamp=2021-05-28T00:02:09.679551Z Entering 2021-05-28T00:02:09.679Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=21127c5f-bab4-4ff4-9daa-1725e8abf87a InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.679477Z InvocationID=128abe25-56d4-490e-bd71-bdc23d91e5ae LogTimestamp=2021-05-28T00:02:09.679551Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:02:09.781Z [qtp580718781-102] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:02:09.680Z InvocationID=5072d3a7-a259-4d16-9b44-2722b89669ee PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@28124d17', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 100ms 2021-05-28T00:02:09.782Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=102 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:02:09.680Z InvocationID=5072d3a7-a259-4d16-9b44-2722b89669ee ErrorCode=409 LogTimestamp=2021-05-28T00:02:09.782044Z ResponseDesc=Conflict Exiting. 2021-05-28T00:02:09.782Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:02:09.878Z [qtp580718781-100] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.878695Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:09.878Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:09.878695Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:09.879Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:02:09.983Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=021fd1af-749f-49df-bb7a-36c4ed608605 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.983652Z InvocationID=fb40f9fa-07b8-48db-a165-63988898764d LogTimestamp=2021-05-28T00:02:09.983688Z Entering 2021-05-28T00:02:09.984Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=021fd1af-749f-49df-bb7a-36c4ed608605 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:09.983652Z InvocationID=fb40f9fa-07b8-48db-a165-63988898764d LogTimestamp=2021-05-28T00:02:09.983688Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:02:10.176Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:02:09.984Z InvocationID=241ebaa6-53d8-407f-9ce9-10807543a987 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@28124d17', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 191ms 2021-05-28T00:02:10.177Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:02:09.984Z InvocationID=241ebaa6-53d8-407f-9ce9-10807543a987 ErrorCode=409 LogTimestamp=2021-05-28T00:02:10.176916Z ResponseDesc=Conflict Exiting. 2021-05-28T00:02:10.177Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:02:10.358Z [qtp580718781-107] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:10.358124Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:10.358Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:10.358124Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:10.359Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:02:10.474Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b8f21365-3643-47e5-a36a-dae6ff16592e InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:10.474423Z InvocationID=158027cf-eb58-43f5-acfe-7797922d9636 LogTimestamp=2021-05-28T00:02:10.474525Z Entering 2021-05-28T00:02:10.474Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b8f21365-3643-47e5-a36a-dae6ff16592e InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:10.474423Z InvocationID=158027cf-eb58-43f5-acfe-7797922d9636 LogTimestamp=2021-05-28T00:02:10.474525Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:02:10.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:02:10.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:02:10.774Z [qtp580718781-108] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:02:10.475Z InvocationID=2917aa1b-c5ae-49c2-b997-7ef10fee79c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@28124d17', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 296ms 2021-05-28T00:02:10.775Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=300 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:02:10.475Z InvocationID=2917aa1b-c5ae-49c2-b997-7ef10fee79c8 ErrorCode=409 LogTimestamp=2021-05-28T00:02:10.775527Z ResponseDesc=Conflict Exiting. 2021-05-28T00:02:10.776Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:02:11.270Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ae1001cd-6ffb-480f-b31c-c5cfb6f58752 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:11.270540Z InvocationID=39564093-0057-40f0-9984-d31b3d0bd396 LogTimestamp=2021-05-28T00:02:11.270606Z Entering 2021-05-28T00:02:11.271Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ae1001cd-6ffb-480f-b31c-c5cfb6f58752 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:11.270540Z InvocationID=39564093-0057-40f0-9984-d31b3d0bd396 LogTimestamp=2021-05-28T00:02:11.270606Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:02:11.383Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=111 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:02:11.271Z InvocationID=aa855eec-5ebc-4038-b675-44a514e9f67b LogTimestamp=2021-05-28T00:02:11.382737Z ResponseDesc=OK Exiting. 2021-05-28T00:02:11.383Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:02:11.573Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:11.573743Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:11.573Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:11.573743Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:11.575Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:02:11.775Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=43022ca9-a6d3-43bb-a419-ff8a5a4c7a63 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:11.775254Z InvocationID=22a1a455-158d-493c-910e-4ee108fdfc5c LogTimestamp=2021-05-28T00:02:11.775315Z Entering 2021-05-28T00:02:11.776Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=43022ca9-a6d3-43bb-a419-ff8a5a4c7a63 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:11.775254Z InvocationID=22a1a455-158d-493c-910e-4ee108fdfc5c LogTimestamp=2021-05-28T00:02:11.775315Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:12.056Z [qtp580718781-107] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:11.776Z InvocationID=d7fc1733-52b1-4570-aae3-7056c586c97b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:02:12.291Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:02:12.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:02:12.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:02:13.857Z [cluster2-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:02:14.457Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2681 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:11.776Z InvocationID=d7fc1733-52b1-4570-aae3-7056c586c97b LogTimestamp=2021-05-28T00:02:14.457586Z ResponseDesc=Created Exiting. 2021-05-28T00:02:14.458Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:14.558Z [qtp580718781-108] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:14.558745Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:14.558Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:14.558745Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:14.559Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:02:14.771Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bcb392c5-2673-4aae-b911-21f0a5ca240d InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:14.771540Z InvocationID=87cb70c3-188a-4601-ad87-37307e362351 LogTimestamp=2021-05-28T00:02:14.771607Z Entering 2021-05-28T00:02:14.772Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bcb392c5-2673-4aae-b911-21f0a5ca240d InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:14.771540Z InvocationID=87cb70c3-188a-4601-ad87-37307e362351 LogTimestamp=2021-05-28T00:02:14.771607Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:15.056Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:14.772Z InvocationID=c8f1a3ad-fcdf-48b9-8378-f88c19bae359 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:02:17.381Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2609 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:14.772Z InvocationID=c8f1a3ad-fcdf-48b9-8378-f88c19bae359 LogTimestamp=2021-05-28T00:02:17.381414Z ResponseDesc=Created Exiting. 2021-05-28T00:02:17.382Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:17.477Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:17.477065Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:17.477Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:17.477065Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:17.478Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:02:17.571Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=534a7910-81f8-42fe-95e0-32b0497953a5 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:17.571429Z InvocationID=c6d9e3ae-86a3-4ccd-98c2-688f9f074f79 LogTimestamp=2021-05-28T00:02:17.571498Z Entering 2021-05-28T00:02:17.571Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=534a7910-81f8-42fe-95e0-32b0497953a5 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:17.571429Z InvocationID=c6d9e3ae-86a3-4ccd-98c2-688f9f074f79 LogTimestamp=2021-05-28T00:02:17.571498Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:17.671Z [qtp580718781-102] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:17.572Z InvocationID=170b40d7-b779-41da-8461-7136bac936c7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:02:19.277Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1704 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:17.572Z InvocationID=170b40d7-b779-41da-8461-7136bac936c7 LogTimestamp=2021-05-28T00:02:19.276890Z ResponseDesc=Created Exiting. 2021-05-28T00:02:19.277Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:19.370Z [qtp580718781-106] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:19.370459Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:19.370Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:19.370459Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:19.371Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:02:19.379Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b40a1589-0fa7-4ee3-8f09-f24d1b4e6f9f InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:19.379683Z InvocationID=a733239b-60ae-4dc2-b78c-de6826c4355a LogTimestamp=2021-05-28T00:02:19.379713Z Entering 2021-05-28T00:02:19.379Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b40a1589-0fa7-4ee3-8f09-f24d1b4e6f9f InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:19.379683Z InvocationID=a733239b-60ae-4dc2-b78c-de6826c4355a LogTimestamp=2021-05-28T00:02:19.379713Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:19.471Z [qtp580718781-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:19.380Z InvocationID=6efe4700-f44a-48c1-9b16-1891480d236a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:02:20.305Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 16000 milliseconds 2021-05-28T00:02:21.668Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2287 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:19.380Z InvocationID=6efe4700-f44a-48c1-9b16-1891480d236a LogTimestamp=2021-05-28T00:02:21.667676Z ResponseDesc=Created Exiting. 2021-05-28T00:02:21.668Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:21.681Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:21.681314Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:21.681Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:21.681314Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:21.681Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:02:21.772Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9371e3a9-a48d-4e0e-a885-c0ae932feab8 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:21.772291Z InvocationID=4d21c281-92f2-4a3d-b42f-a411fe05bc45 LogTimestamp=2021-05-28T00:02:21.772330Z Entering 2021-05-28T00:02:21.772Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9371e3a9-a48d-4e0e-a885-c0ae932feab8 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:21.772291Z InvocationID=4d21c281-92f2-4a3d-b42f-a411fe05bc45 LogTimestamp=2021-05-28T00:02:21.772330Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:21.873Z [qtp580718781-102] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:21.772Z InvocationID=0445f4e9-bf81-4a2c-9e59-8d1c79fddeb6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:02:24.567Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2795 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:21.772Z InvocationID=0445f4e9-bf81-4a2c-9e59-8d1c79fddeb6 LogTimestamp=2021-05-28T00:02:24.567451Z ResponseDesc=Created Exiting. 2021-05-28T00:02:24.568Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:24.769Z [qtp580718781-107] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:24.769045Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:24.769Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:24.769045Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:24.769Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:02:24.957Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=75e014a1-4c2d-4276-9476-516d509a6b84 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:24.957333Z InvocationID=61732a01-706c-4384-ba44-d09ed3901b4d LogTimestamp=2021-05-28T00:02:24.957370Z Entering 2021-05-28T00:02:24.957Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=75e014a1-4c2d-4276-9476-516d509a6b84 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:24.957333Z InvocationID=61732a01-706c-4384-ba44-d09ed3901b4d LogTimestamp=2021-05-28T00:02:24.957370Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:24.971Z [qtp580718781-100] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:24.957Z InvocationID=b152ed32-7f12-4c4b-ab2b-5d4e513ffc0f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:02:25.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:02:25.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:02:26.404Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:02:26.405Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430825b0-da74-4a77-ab1f-86cf4d9f813a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.405Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430825b0-da74-4a77-ab1f-86cf4d9f813a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6788e8d9 2021-05-28T00:02:26.405Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430825b0-da74-4a77-ab1f-86cf4d9f813a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.405Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430825b0-da74-4a77-ab1f-86cf4d9f813a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.411Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd0cc068-633e-4bc4-be21-8a84e0734fa2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.411Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd0cc068-633e-4bc4-be21-8a84e0734fa2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65326664 2021-05-28T00:02:26.411Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd0cc068-633e-4bc4-be21-8a84e0734fa2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.411Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd0cc068-633e-4bc4-be21-8a84e0734fa2 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.413Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e98b4e-ac97-4071-a9f0-5d8d9ea679bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.413Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e98b4e-ac97-4071-a9f0-5d8d9ea679bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@499b39c4 2021-05-28T00:02:26.413Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e98b4e-ac97-4071-a9f0-5d8d9ea679bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.413Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e98b4e-ac97-4071-a9f0-5d8d9ea679bb PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.416Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24cf9a2a-16f3-428b-8c40-389fb1d5c190 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.416Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24cf9a2a-16f3-428b-8c40-389fb1d5c190 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12bd4860 2021-05-28T00:02:26.416Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24cf9a2a-16f3-428b-8c40-389fb1d5c190 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.416Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24cf9a2a-16f3-428b-8c40-389fb1d5c190 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.418Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac73438b-6305-4306-9034-fe599dcf94b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.419Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac73438b-6305-4306-9034-fe599dcf94b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3253a4ce 2021-05-28T00:02:26.419Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac73438b-6305-4306-9034-fe599dcf94b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.419Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac73438b-6305-4306-9034-fe599dcf94b8 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.421Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=297ea1ad-1aa6-4cdc-8d94-a1e92eb12633 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.421Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=297ea1ad-1aa6-4cdc-8d94-a1e92eb12633 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4424e5a6 2021-05-28T00:02:26.421Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=297ea1ad-1aa6-4cdc-8d94-a1e92eb12633 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.421Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=297ea1ad-1aa6-4cdc-8d94-a1e92eb12633 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.424Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c310b9c8-3578-4dad-bcde-ef49df901a63 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.424Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c310b9c8-3578-4dad-bcde-ef49df901a63 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54e1a1ff 2021-05-28T00:02:26.424Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c310b9c8-3578-4dad-bcde-ef49df901a63 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.424Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c310b9c8-3578-4dad-bcde-ef49df901a63 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29ca0a08-c708-42d3-89d7-e4d917ba5828 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29ca0a08-c708-42d3-89d7-e4d917ba5828 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ce70d4b 2021-05-28T00:02:26.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29ca0a08-c708-42d3-89d7-e4d917ba5828 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.426Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29ca0a08-c708-42d3-89d7-e4d917ba5828 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4aba61d5-2a91-4930-9569-bb3802cd973a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4aba61d5-2a91-4930-9569-bb3802cd973a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33da9cd 2021-05-28T00:02:26.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4aba61d5-2a91-4930-9569-bb3802cd973a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.428Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4aba61d5-2a91-4930-9569-bb3802cd973a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17888014-9c16-4d30-b30d-7f7fe75a5fda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17888014-9c16-4d30-b30d-7f7fe75a5fda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@605765d9 2021-05-28T00:02:26.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17888014-9c16-4d30-b30d-7f7fe75a5fda PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.430Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17888014-9c16-4d30-b30d-7f7fe75a5fda PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7642d184-735c-4853-93fc-f0c33ddb1675 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7642d184-735c-4853-93fc-f0c33ddb1675 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3960ce75 2021-05-28T00:02:26.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7642d184-735c-4853-93fc-f0c33ddb1675 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.432Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7642d184-735c-4853-93fc-f0c33ddb1675 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.434Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3016227e-dc31-4bbb-97d5-5cc72143be70 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3016227e-dc31-4bbb-97d5-5cc72143be70 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2df91147 2021-05-28T00:02:26.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3016227e-dc31-4bbb-97d5-5cc72143be70 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.435Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3016227e-dc31-4bbb-97d5-5cc72143be70 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13203dc-4031-4772-88f7-a0de59cf1050 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13203dc-4031-4772-88f7-a0de59cf1050 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43e4a6f4 2021-05-28T00:02:26.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13203dc-4031-4772-88f7-a0de59cf1050 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.437Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c13203dc-4031-4772-88f7-a0de59cf1050 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4303f9db-2dcb-4e26-bad9-16df0c7d9a12 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4303f9db-2dcb-4e26-bad9-16df0c7d9a12 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18c67afc 2021-05-28T00:02:26.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4303f9db-2dcb-4e26-bad9-16df0c7d9a12 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.439Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4303f9db-2dcb-4e26-bad9-16df0c7d9a12 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=967a0d91-0041-4c18-8218-332e822b7a9a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=967a0d91-0041-4c18-8218-332e822b7a9a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27ea0cf7 2021-05-28T00:02:26.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=967a0d91-0041-4c18-8218-332e822b7a9a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.441Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=967a0d91-0041-4c18-8218-332e822b7a9a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.442Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=233c1bef-c6d5-46f3-8a96-5598b0746a8d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=233c1bef-c6d5-46f3-8a96-5598b0746a8d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@427c5264 2021-05-28T00:02:26.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=233c1bef-c6d5-46f3-8a96-5598b0746a8d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.443Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=233c1bef-c6d5-46f3-8a96-5598b0746a8d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0310ba9e-a321-4142-8835-8bd760f01b59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0310ba9e-a321-4142-8835-8bd760f01b59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24c99fd9 2021-05-28T00:02:26.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0310ba9e-a321-4142-8835-8bd760f01b59 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.444Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0310ba9e-a321-4142-8835-8bd760f01b59 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a788c1-a693-42c3-9400-6909839b75e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a788c1-a693-42c3-9400-6909839b75e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ff30026 2021-05-28T00:02:26.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a788c1-a693-42c3-9400-6909839b75e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.446Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a788c1-a693-42c3-9400-6909839b75e3 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae40c04-866e-4542-b4b9-6c0248d8b2ae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae40c04-866e-4542-b4b9-6c0248d8b2ae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7809289b 2021-05-28T00:02:26.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae40c04-866e-4542-b4b9-6c0248d8b2ae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ae40c04-866e-4542-b4b9-6c0248d8b2ae PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97e2247a-9a1d-4059-a41e-9200119886db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97e2247a-9a1d-4059-a41e-9200119886db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58c6aa8c 2021-05-28T00:02:26.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97e2247a-9a1d-4059-a41e-9200119886db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97e2247a-9a1d-4059-a41e-9200119886db PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=feebd842-4792-40c8-88c2-a0e88c8dadbc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=feebd842-4792-40c8-88c2-a0e88c8dadbc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3019d11a 2021-05-28T00:02:26.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=feebd842-4792-40c8-88c2-a0e88c8dadbc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=feebd842-4792-40c8-88c2-a0e88c8dadbc PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5816457b-ecc0-4dcc-88fb-404c92b053ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5816457b-ecc0-4dcc-88fb-404c92b053ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27fc150f 2021-05-28T00:02:26.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5816457b-ecc0-4dcc-88fb-404c92b053ce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.454Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5816457b-ecc0-4dcc-88fb-404c92b053ce PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4a2c1f-7c1d-4683-aff6-08b17dd1b7d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4a2c1f-7c1d-4683-aff6-08b17dd1b7d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1600ac9b 2021-05-28T00:02:26.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4a2c1f-7c1d-4683-aff6-08b17dd1b7d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4a2c1f-7c1d-4683-aff6-08b17dd1b7d1 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6868cd-e0fb-4575-8da5-1e2258b9443d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6868cd-e0fb-4575-8da5-1e2258b9443d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b7fe61d 2021-05-28T00:02:26.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6868cd-e0fb-4575-8da5-1e2258b9443d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6868cd-e0fb-4575-8da5-1e2258b9443d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4fa4e4d-cdc8-4c65-9eb2-3eca03d9e3e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4fa4e4d-cdc8-4c65-9eb2-3eca03d9e3e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5107969 2021-05-28T00:02:26.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4fa4e4d-cdc8-4c65-9eb2-3eca03d9e3e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4fa4e4d-cdc8-4c65-9eb2-3eca03d9e3e5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0197baf1-2f80-4d54-b0a1-611a3c39f47f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0197baf1-2f80-4d54-b0a1-611a3c39f47f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@428350a9 2021-05-28T00:02:26.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0197baf1-2f80-4d54-b0a1-611a3c39f47f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0197baf1-2f80-4d54-b0a1-611a3c39f47f PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c52be85-157a-4944-aa32-c403405c8846 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c52be85-157a-4944-aa32-c403405c8846 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51744816 2021-05-28T00:02:26.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c52be85-157a-4944-aa32-c403405c8846 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c52be85-157a-4944-aa32-c403405c8846 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abd8529-0a41-41b4-bd17-0462b582a6b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abd8529-0a41-41b4-bd17-0462b582a6b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55bf132b 2021-05-28T00:02:26.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abd8529-0a41-41b4-bd17-0462b582a6b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abd8529-0a41-41b4-bd17-0462b582a6b9 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f46a89da-cd37-4b53-acdc-28935d4de565 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f46a89da-cd37-4b53-acdc-28935d4de565 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5362f3c4 2021-05-28T00:02:26.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f46a89da-cd37-4b53-acdc-28935d4de565 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f46a89da-cd37-4b53-acdc-28935d4de565 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba952d6-a4d2-4274-b892-6b5fb1a6ae90 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba952d6-a4d2-4274-b892-6b5fb1a6ae90 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b3ce719 2021-05-28T00:02:26.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba952d6-a4d2-4274-b892-6b5fb1a6ae90 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba952d6-a4d2-4274-b892-6b5fb1a6ae90 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.473Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1515 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:24.957Z InvocationID=b152ed32-7f12-4c4b-ab2b-5d4e513ffc0f LogTimestamp=2021-05-28T00:02:26.472809Z ResponseDesc=Created Exiting. 2021-05-28T00:02:26.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=060f6122-0f37-4302-902a-2c476807d2da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=060f6122-0f37-4302-902a-2c476807d2da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@787b2a90 2021-05-28T00:02:26.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=060f6122-0f37-4302-902a-2c476807d2da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=060f6122-0f37-4302-902a-2c476807d2da PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.473Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:26.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f40270c-6875-4c30-a996-a44aa72bb051 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f40270c-6875-4c30-a996-a44aa72bb051 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55f048ff 2021-05-28T00:02:26.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f40270c-6875-4c30-a996-a44aa72bb051 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f40270c-6875-4c30-a996-a44aa72bb051 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fb80fd5-20d7-438d-a0d8-8f506353eb58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fb80fd5-20d7-438d-a0d8-8f506353eb58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11487d56 2021-05-28T00:02:26.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fb80fd5-20d7-438d-a0d8-8f506353eb58 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fb80fd5-20d7-438d-a0d8-8f506353eb58 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e7f455-88b9-401c-b239-cdc3b9fdf210 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e7f455-88b9-401c-b239-cdc3b9fdf210 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7da04449 2021-05-28T00:02:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e7f455-88b9-401c-b239-cdc3b9fdf210 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e7f455-88b9-401c-b239-cdc3b9fdf210 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0497608-1dae-4ab0-a0da-b168ffa1ccf6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0497608-1dae-4ab0-a0da-b168ffa1ccf6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c2ecec7 2021-05-28T00:02:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0497608-1dae-4ab0-a0da-b168ffa1ccf6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0497608-1dae-4ab0-a0da-b168ffa1ccf6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58eb2e00-9c8d-4e73-806f-5237c91de45a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58eb2e00-9c8d-4e73-806f-5237c91de45a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bced20d 2021-05-28T00:02:26.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58eb2e00-9c8d-4e73-806f-5237c91de45a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58eb2e00-9c8d-4e73-806f-5237c91de45a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47df1ece-b931-473a-b657-b78c36fa5cbd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47df1ece-b931-473a-b657-b78c36fa5cbd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59beacef 2021-05-28T00:02:26.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47df1ece-b931-473a-b657-b78c36fa5cbd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47df1ece-b931-473a-b657-b78c36fa5cbd PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0bc9d60-531e-48ef-bb4f-a37f03cf97ac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0bc9d60-531e-48ef-bb4f-a37f03cf97ac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2643299d 2021-05-28T00:02:26.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0bc9d60-531e-48ef-bb4f-a37f03cf97ac PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0bc9d60-531e-48ef-bb4f-a37f03cf97ac PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.488Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3790de95-fe23-435c-a7cb-15aa6fb099f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.488Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3790de95-fe23-435c-a7cb-15aa6fb099f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cb18364 2021-05-28T00:02:26.488Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3790de95-fe23-435c-a7cb-15aa6fb099f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.488Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3790de95-fe23-435c-a7cb-15aa6fb099f6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.489Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aee2f18-074a-4de9-ba9e-2bdba2b72bf9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.490Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aee2f18-074a-4de9-ba9e-2bdba2b72bf9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71127645 2021-05-28T00:02:26.490Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aee2f18-074a-4de9-ba9e-2bdba2b72bf9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.490Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6aee2f18-074a-4de9-ba9e-2bdba2b72bf9 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.491Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59bf410c-fdcd-4b8f-a7c5-c79d855d9f99 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.491Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59bf410c-fdcd-4b8f-a7c5-c79d855d9f99 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d67755c 2021-05-28T00:02:26.491Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59bf410c-fdcd-4b8f-a7c5-c79d855d9f99 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.492Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59bf410c-fdcd-4b8f-a7c5-c79d855d9f99 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fd7a11c-b124-4fc9-b1d9-604b5bc6dc2a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fd7a11c-b124-4fc9-b1d9-604b5bc6dc2a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6257a0ac 2021-05-28T00:02:26.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fd7a11c-b124-4fc9-b1d9-604b5bc6dc2a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fd7a11c-b124-4fc9-b1d9-604b5bc6dc2a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.496Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9225dacd-bc60-4a20-b672-900ffc16afed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.496Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9225dacd-bc60-4a20-b672-900ffc16afed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62fdb521 2021-05-28T00:02:26.496Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9225dacd-bc60-4a20-b672-900ffc16afed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.496Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9225dacd-bc60-4a20-b672-900ffc16afed PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.498Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fd0c80b-b739-4ec7-9eb5-7d3b8439d120 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.498Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fd0c80b-b739-4ec7-9eb5-7d3b8439d120 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bf5765 2021-05-28T00:02:26.498Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fd0c80b-b739-4ec7-9eb5-7d3b8439d120 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.498Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fd0c80b-b739-4ec7-9eb5-7d3b8439d120 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.500Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e848cbcd-2ab5-4e4c-a82c-ff8bcc2989f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.500Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e848cbcd-2ab5-4e4c-a82c-ff8bcc2989f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66940fbb 2021-05-28T00:02:26.500Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e848cbcd-2ab5-4e4c-a82c-ff8bcc2989f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.500Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e848cbcd-2ab5-4e4c-a82c-ff8bcc2989f4 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a60123d-fd50-4f58-a249-a132de5fbef5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a60123d-fd50-4f58-a249-a132de5fbef5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39998105 2021-05-28T00:02:26.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a60123d-fd50-4f58-a249-a132de5fbef5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a60123d-fd50-4f58-a249-a132de5fbef5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.505Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5af720f6-5a78-4a00-9496-4330f00b707e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.505Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5af720f6-5a78-4a00-9496-4330f00b707e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36c27faa 2021-05-28T00:02:26.505Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5af720f6-5a78-4a00-9496-4330f00b707e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.505Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5af720f6-5a78-4a00-9496-4330f00b707e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.508Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37d4453-d799-44bf-8a06-c25e2f9b7d65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.508Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37d4453-d799-44bf-8a06-c25e2f9b7d65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c3c1268 2021-05-28T00:02:26.508Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37d4453-d799-44bf-8a06-c25e2f9b7d65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.508Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e37d4453-d799-44bf-8a06-c25e2f9b7d65 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.510Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0f2df-feaa-47af-ae14-9732f2a3f71c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.510Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0f2df-feaa-47af-ae14-9732f2a3f71c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a932583 2021-05-28T00:02:26.510Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0f2df-feaa-47af-ae14-9732f2a3f71c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.510Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08e0f2df-feaa-47af-ae14-9732f2a3f71c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.512Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c388e-bdcc-46c9-9cc7-3bc1fb5af942 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.512Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c388e-bdcc-46c9-9cc7-3bc1fb5af942 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f771c7d 2021-05-28T00:02:26.512Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c388e-bdcc-46c9-9cc7-3bc1fb5af942 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.512Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c388e-bdcc-46c9-9cc7-3bc1fb5af942 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.514Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf645ab0-35c2-46af-afd9-c84290db06e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.514Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf645ab0-35c2-46af-afd9-c84290db06e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5aed0316 2021-05-28T00:02:26.514Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf645ab0-35c2-46af-afd9-c84290db06e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.514Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf645ab0-35c2-46af-afd9-c84290db06e9 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.516Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f32975e5-6b74-4b60-9bb9-73d3d3d8b9f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.516Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f32975e5-6b74-4b60-9bb9-73d3d3d8b9f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76eee571 2021-05-28T00:02:26.516Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f32975e5-6b74-4b60-9bb9-73d3d3d8b9f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.516Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f32975e5-6b74-4b60-9bb9-73d3d3d8b9f6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.519Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8c7ff2-053d-4147-a38f-3a3d5267f8cc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.519Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8c7ff2-053d-4147-a38f-3a3d5267f8cc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65c26912 2021-05-28T00:02:26.519Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8c7ff2-053d-4147-a38f-3a3d5267f8cc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.519Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8c7ff2-053d-4147-a38f-3a3d5267f8cc PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ce39d8f-4184-480c-a8ca-cf4686a90899 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ce39d8f-4184-480c-a8ca-cf4686a90899 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22911523 2021-05-28T00:02:26.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ce39d8f-4184-480c-a8ca-cf4686a90899 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.521Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ce39d8f-4184-480c-a8ca-cf4686a90899 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.523Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69bc5f05-e59c-4ef2-a7f5-fbf98cac7262 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.523Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69bc5f05-e59c-4ef2-a7f5-fbf98cac7262 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10d657d4 2021-05-28T00:02:26.523Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69bc5f05-e59c-4ef2-a7f5-fbf98cac7262 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.523Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69bc5f05-e59c-4ef2-a7f5-fbf98cac7262 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.525Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=213160e3-adae-4ef3-a1fb-43f45eceb083 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.525Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=213160e3-adae-4ef3-a1fb-43f45eceb083 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1112edd2 2021-05-28T00:02:26.525Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=213160e3-adae-4ef3-a1fb-43f45eceb083 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.525Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=213160e3-adae-4ef3-a1fb-43f45eceb083 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8b1cae-e6a2-449c-b285-a9c916542449 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8b1cae-e6a2-449c-b285-a9c916542449 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@676e8c1b 2021-05-28T00:02:26.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8b1cae-e6a2-449c-b285-a9c916542449 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.526Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8b1cae-e6a2-449c-b285-a9c916542449 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab4db44-4936-4080-82a4-f3304e7d8fc0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab4db44-4936-4080-82a4-f3304e7d8fc0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4dfcb5d8 2021-05-28T00:02:26.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab4db44-4936-4080-82a4-f3304e7d8fc0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.528Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab4db44-4936-4080-82a4-f3304e7d8fc0 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bf28ac6-3a36-4557-9cd5-e9b2d64c83d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bf28ac6-3a36-4557-9cd5-e9b2d64c83d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fb63344 2021-05-28T00:02:26.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bf28ac6-3a36-4557-9cd5-e9b2d64c83d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.530Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bf28ac6-3a36-4557-9cd5-e9b2d64c83d1 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7554d043-8d71-4fe0-b7c2-4801d1c5a6cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7554d043-8d71-4fe0-b7c2-4801d1c5a6cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69cc2f1d 2021-05-28T00:02:26.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7554d043-8d71-4fe0-b7c2-4801d1c5a6cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.532Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7554d043-8d71-4fe0-b7c2-4801d1c5a6cf PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a75ff079-698d-49e6-b4c4-cbc0716635de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a75ff079-698d-49e6-b4c4-cbc0716635de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ad1686b 2021-05-28T00:02:26.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a75ff079-698d-49e6-b4c4-cbc0716635de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.533Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a75ff079-698d-49e6-b4c4-cbc0716635de PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703f366e-4274-4f6f-95d1-ef52794cfd48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703f366e-4274-4f6f-95d1-ef52794cfd48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52ab509c 2021-05-28T00:02:26.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703f366e-4274-4f6f-95d1-ef52794cfd48 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.536Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=703f366e-4274-4f6f-95d1-ef52794cfd48 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88a4b16-54ad-4493-9fad-b78fad213de4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88a4b16-54ad-4493-9fad-b78fad213de4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cd9759e 2021-05-28T00:02:26.537Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88a4b16-54ad-4493-9fad-b78fad213de4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.538Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88a4b16-54ad-4493-9fad-b78fad213de4 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a11c22-637f-48e9-aa12-58954c0addaf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a11c22-637f-48e9-aa12-58954c0addaf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e517df 2021-05-28T00:02:26.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a11c22-637f-48e9-aa12-58954c0addaf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a11c22-637f-48e9-aa12-58954c0addaf PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ecf615-28e4-4768-94f9-8068c0a6564e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ecf615-28e4-4768-94f9-8068c0a6564e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7506b518 2021-05-28T00:02:26.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ecf615-28e4-4768-94f9-8068c0a6564e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.541Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6ecf615-28e4-4768-94f9-8068c0a6564e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d93b91c8-a842-4bbb-9ac1-e473e7c7e859 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d93b91c8-a842-4bbb-9ac1-e473e7c7e859 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7da12494 2021-05-28T00:02:26.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d93b91c8-a842-4bbb-9ac1-e473e7c7e859 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.543Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d93b91c8-a842-4bbb-9ac1-e473e7c7e859 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe5285b-7b1a-47d6-ae67-516fa7997004 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe5285b-7b1a-47d6-ae67-516fa7997004 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a93cb3b 2021-05-28T00:02:26.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe5285b-7b1a-47d6-ae67-516fa7997004 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbe5285b-7b1a-47d6-ae67-516fa7997004 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe8dd10-3610-408e-b22d-81d244ea9efc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe8dd10-3610-408e-b22d-81d244ea9efc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16e46859 2021-05-28T00:02:26.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe8dd10-3610-408e-b22d-81d244ea9efc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe8dd10-3610-408e-b22d-81d244ea9efc PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c537c84c-e0a8-4348-88e7-8921bb51d320 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c537c84c-e0a8-4348-88e7-8921bb51d320 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@560aa1e8 2021-05-28T00:02:26.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c537c84c-e0a8-4348-88e7-8921bb51d320 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c537c84c-e0a8-4348-88e7-8921bb51d320 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=289d710d-61fc-4117-9fb0-82778827cc40 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=289d710d-61fc-4117-9fb0-82778827cc40 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@227bba39 2021-05-28T00:02:26.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=289d710d-61fc-4117-9fb0-82778827cc40 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=289d710d-61fc-4117-9fb0-82778827cc40 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7510b71-30d4-44cd-90a9-86cb85af61a2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7510b71-30d4-44cd-90a9-86cb85af61a2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@695675e 2021-05-28T00:02:26.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7510b71-30d4-44cd-90a9-86cb85af61a2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7510b71-30d4-44cd-90a9-86cb85af61a2 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1f849c-1a5b-4c47-9c8a-488998652f57 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1f849c-1a5b-4c47-9c8a-488998652f57 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33866175 2021-05-28T00:02:26.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1f849c-1a5b-4c47-9c8a-488998652f57 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a1f849c-1a5b-4c47-9c8a-488998652f57 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11dab1bb-2b44-422c-bb52-be6875c29728 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11dab1bb-2b44-422c-bb52-be6875c29728 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3000930c 2021-05-28T00:02:26.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11dab1bb-2b44-422c-bb52-be6875c29728 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11dab1bb-2b44-422c-bb52-be6875c29728 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.557Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:26.557371Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:26.557Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:26.557371Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:26.557Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:02:26.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9dc716f-da2e-474d-86ca-04f29894ee94 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9dc716f-da2e-474d-86ca-04f29894ee94 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f554d47 2021-05-28T00:02:26.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9dc716f-da2e-474d-86ca-04f29894ee94 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9dc716f-da2e-474d-86ca-04f29894ee94 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5219c590-faa0-4b9b-8fb7-16896ac57d47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5219c590-faa0-4b9b-8fb7-16896ac57d47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62576c64 2021-05-28T00:02:26.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5219c590-faa0-4b9b-8fb7-16896ac57d47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5219c590-faa0-4b9b-8fb7-16896ac57d47 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f94b74e-5531-40b8-8f3c-37639bf29183 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f94b74e-5531-40b8-8f3c-37639bf29183 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@625a2492 2021-05-28T00:02:26.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f94b74e-5531-40b8-8f3c-37639bf29183 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f94b74e-5531-40b8-8f3c-37639bf29183 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=deff267d-e93d-4ee1-af1b-5b8edbf2e76b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=deff267d-e93d-4ee1-af1b-5b8edbf2e76b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56f65326 2021-05-28T00:02:26.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=deff267d-e93d-4ee1-af1b-5b8edbf2e76b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=deff267d-e93d-4ee1-af1b-5b8edbf2e76b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9983f2cc-a1eb-46b5-aa00-e0066409d931 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9983f2cc-a1eb-46b5-aa00-e0066409d931 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33ac03ca 2021-05-28T00:02:26.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9983f2cc-a1eb-46b5-aa00-e0066409d931 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9983f2cc-a1eb-46b5-aa00-e0066409d931 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8df7a92-18d7-4f6b-aecb-d03bfe9e5553 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8df7a92-18d7-4f6b-aecb-d03bfe9e5553 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a416d97 2021-05-28T00:02:26.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8df7a92-18d7-4f6b-aecb-d03bfe9e5553 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8df7a92-18d7-4f6b-aecb-d03bfe9e5553 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.568Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=97e02840-091f-4352-bef2-32d89d36ed08 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:26.568569Z InvocationID=07e5f6b2-1542-44c6-a148-0d14967e4e8e LogTimestamp=2021-05-28T00:02:26.568628Z Entering 2021-05-28T00:02:26.569Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=97e02840-091f-4352-bef2-32d89d36ed08 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:26.568569Z InvocationID=07e5f6b2-1542-44c6-a148-0d14967e4e8e LogTimestamp=2021-05-28T00:02:26.568628Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:26.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34d06f19-a976-4575-8761-29f51411bc26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34d06f19-a976-4575-8761-29f51411bc26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@590caaf9 2021-05-28T00:02:26.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34d06f19-a976-4575-8761-29f51411bc26 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34d06f19-a976-4575-8761-29f51411bc26 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae132e7-f73c-460e-8600-ae6425d1d602 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae132e7-f73c-460e-8600-ae6425d1d602 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d07ad9d 2021-05-28T00:02:26.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae132e7-f73c-460e-8600-ae6425d1d602 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae132e7-f73c-460e-8600-ae6425d1d602 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f101d6a5-bdde-497b-9662-73e2af338145 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f101d6a5-bdde-497b-9662-73e2af338145 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e664ba4 2021-05-28T00:02:26.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f101d6a5-bdde-497b-9662-73e2af338145 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f101d6a5-bdde-497b-9662-73e2af338145 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7142a672-5eec-4082-a83c-8e92d696787b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7142a672-5eec-4082-a83c-8e92d696787b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f2aae2a 2021-05-28T00:02:26.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7142a672-5eec-4082-a83c-8e92d696787b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7142a672-5eec-4082-a83c-8e92d696787b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a21c899-6c18-4219-be5f-139fe76da443 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a21c899-6c18-4219-be5f-139fe76da443 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@334d8654 2021-05-28T00:02:26.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a21c899-6c18-4219-be5f-139fe76da443 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a21c899-6c18-4219-be5f-139fe76da443 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84a7beca-d49a-4f3b-9ca3-f4445f100c4f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84a7beca-d49a-4f3b-9ca3-f4445f100c4f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3326bee5 2021-05-28T00:02:26.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84a7beca-d49a-4f3b-9ca3-f4445f100c4f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84a7beca-d49a-4f3b-9ca3-f4445f100c4f PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.578Z [qtp580718781-108] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:26.569Z InvocationID=0775d6fc-6a50-44cb-aeca-713ed4d5462c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6d8d434-3e1a-4122-9623-4c4766f8077e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6d8d434-3e1a-4122-9623-4c4766f8077e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fc1f84e 2021-05-28T00:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6d8d434-3e1a-4122-9623-4c4766f8077e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6d8d434-3e1a-4122-9623-4c4766f8077e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e4a3587-32d4-4707-998c-81bdbafac934 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e4a3587-32d4-4707-998c-81bdbafac934 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35405062 2021-05-28T00:02:26.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e4a3587-32d4-4707-998c-81bdbafac934 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e4a3587-32d4-4707-998c-81bdbafac934 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83141271-6b4a-4921-8ffa-3ee23b6af0a6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83141271-6b4a-4921-8ffa-3ee23b6af0a6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ae01fb8 2021-05-28T00:02:26.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83141271-6b4a-4921-8ffa-3ee23b6af0a6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83141271-6b4a-4921-8ffa-3ee23b6af0a6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fba2d77b-a163-450c-80dd-632ee6c0018e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fba2d77b-a163-450c-80dd-632ee6c0018e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cb75597 2021-05-28T00:02:26.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fba2d77b-a163-450c-80dd-632ee6c0018e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fba2d77b-a163-450c-80dd-632ee6c0018e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01fed312-a78a-4133-8831-99765e1e047b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01fed312-a78a-4133-8831-99765e1e047b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cd4c3c7 2021-05-28T00:02:26.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01fed312-a78a-4133-8831-99765e1e047b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01fed312-a78a-4133-8831-99765e1e047b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18795da3-837c-42cc-94b5-a3f27df10932 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18795da3-837c-42cc-94b5-a3f27df10932 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ba0517f 2021-05-28T00:02:26.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18795da3-837c-42cc-94b5-a3f27df10932 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18795da3-837c-42cc-94b5-a3f27df10932 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa788830-1ef8-4de6-ad64-a88eafabc726 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa788830-1ef8-4de6-ad64-a88eafabc726 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51d7946c 2021-05-28T00:02:26.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa788830-1ef8-4de6-ad64-a88eafabc726 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa788830-1ef8-4de6-ad64-a88eafabc726 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ca7f091-f6cb-4e8c-b32b-74f20e298b52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ca7f091-f6cb-4e8c-b32b-74f20e298b52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79a682c 2021-05-28T00:02:26.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ca7f091-f6cb-4e8c-b32b-74f20e298b52 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ca7f091-f6cb-4e8c-b32b-74f20e298b52 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b61ec9-5221-4258-8d22-83a48b04b687 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b61ec9-5221-4258-8d22-83a48b04b687 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d7a59e3 2021-05-28T00:02:26.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b61ec9-5221-4258-8d22-83a48b04b687 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b61ec9-5221-4258-8d22-83a48b04b687 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d945c08e-698d-4980-b8fc-c3777941acd5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d945c08e-698d-4980-b8fc-c3777941acd5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1468de9a 2021-05-28T00:02:26.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d945c08e-698d-4980-b8fc-c3777941acd5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d945c08e-698d-4980-b8fc-c3777941acd5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fa76618-b5d8-4836-b05a-623319e2a359 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fa76618-b5d8-4836-b05a-623319e2a359 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79f15c8f 2021-05-28T00:02:26.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fa76618-b5d8-4836-b05a-623319e2a359 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fa76618-b5d8-4836-b05a-623319e2a359 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b907883f-834b-4455-b914-a1844037ffeb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b907883f-834b-4455-b914-a1844037ffeb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2dfe9095 2021-05-28T00:02:26.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b907883f-834b-4455-b914-a1844037ffeb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b907883f-834b-4455-b914-a1844037ffeb PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13644130-1096-4e68-966a-4e1debe37a19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13644130-1096-4e68-966a-4e1debe37a19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52b46188 2021-05-28T00:02:26.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13644130-1096-4e68-966a-4e1debe37a19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13644130-1096-4e68-966a-4e1debe37a19 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdd4536-5afd-4117-943a-2d2d9c6e80a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdd4536-5afd-4117-943a-2d2d9c6e80a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ae94a 2021-05-28T00:02:26.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdd4536-5afd-4117-943a-2d2d9c6e80a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdd4536-5afd-4117-943a-2d2d9c6e80a5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daaf52d9-074c-466c-9436-597ccf8a4453 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daaf52d9-074c-466c-9436-597ccf8a4453 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@731cc8ee 2021-05-28T00:02:26.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daaf52d9-074c-466c-9436-597ccf8a4453 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daaf52d9-074c-466c-9436-597ccf8a4453 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6510e315-f149-4a65-97a5-9b4fe74e1721 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6510e315-f149-4a65-97a5-9b4fe74e1721 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2abf9420 2021-05-28T00:02:26.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6510e315-f149-4a65-97a5-9b4fe74e1721 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6510e315-f149-4a65-97a5-9b4fe74e1721 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0ecfab-e4e5-4528-859a-6f5c9bb48ab8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0ecfab-e4e5-4528-859a-6f5c9bb48ab8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b7608de 2021-05-28T00:02:26.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0ecfab-e4e5-4528-859a-6f5c9bb48ab8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0ecfab-e4e5-4528-859a-6f5c9bb48ab8 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5aada05-caab-4280-b6a8-c9b7f226f654 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5aada05-caab-4280-b6a8-c9b7f226f654 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c118a43 2021-05-28T00:02:26.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5aada05-caab-4280-b6a8-c9b7f226f654 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5aada05-caab-4280-b6a8-c9b7f226f654 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c9a21a6-c6f1-4f2d-a04f-ccae20b4146b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c9a21a6-c6f1-4f2d-a04f-ccae20b4146b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@195f6897 2021-05-28T00:02:26.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c9a21a6-c6f1-4f2d-a04f-ccae20b4146b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c9a21a6-c6f1-4f2d-a04f-ccae20b4146b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965c80d6-3b76-48a4-89ba-3b633db0ac64 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965c80d6-3b76-48a4-89ba-3b633db0ac64 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f97e108 2021-05-28T00:02:26.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965c80d6-3b76-48a4-89ba-3b633db0ac64 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965c80d6-3b76-48a4-89ba-3b633db0ac64 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a78a0ea-927a-485c-92ea-a31b92a4bfda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a78a0ea-927a-485c-92ea-a31b92a4bfda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@244c30dc 2021-05-28T00:02:26.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a78a0ea-927a-485c-92ea-a31b92a4bfda PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a78a0ea-927a-485c-92ea-a31b92a4bfda PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6636adeb-0c54-41f2-80a0-e8e192fd95cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6636adeb-0c54-41f2-80a0-e8e192fd95cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@248fe584 2021-05-28T00:02:26.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6636adeb-0c54-41f2-80a0-e8e192fd95cd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6636adeb-0c54-41f2-80a0-e8e192fd95cd PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37c8e120-4cb6-4c88-84ac-254ca838b13a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37c8e120-4cb6-4c88-84ac-254ca838b13a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46763a52 2021-05-28T00:02:26.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37c8e120-4cb6-4c88-84ac-254ca838b13a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37c8e120-4cb6-4c88-84ac-254ca838b13a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cab1d8-577f-473b-b4d2-49fcc413543b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cab1d8-577f-473b-b4d2-49fcc413543b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36990c35 2021-05-28T00:02:26.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cab1d8-577f-473b-b4d2-49fcc413543b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cab1d8-577f-473b-b4d2-49fcc413543b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c722b64-2692-4837-9c2a-cad6aa58ff99 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c722b64-2692-4837-9c2a-cad6aa58ff99 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@731610ea 2021-05-28T00:02:26.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c722b64-2692-4837-9c2a-cad6aa58ff99 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c722b64-2692-4837-9c2a-cad6aa58ff99 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4b7b858-8fd7-4d37-a994-aa1b8c95c38c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4b7b858-8fd7-4d37-a994-aa1b8c95c38c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@286e78c9 2021-05-28T00:02:26.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4b7b858-8fd7-4d37-a994-aa1b8c95c38c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4b7b858-8fd7-4d37-a994-aa1b8c95c38c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=041be9cf-9fc0-4ca6-bf3d-5bf1beb43a36 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=041be9cf-9fc0-4ca6-bf3d-5bf1beb43a36 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f3d633c 2021-05-28T00:02:26.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=041be9cf-9fc0-4ca6-bf3d-5bf1beb43a36 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=041be9cf-9fc0-4ca6-bf3d-5bf1beb43a36 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc59b6ad-59eb-4c41-b988-61104076666b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc59b6ad-59eb-4c41-b988-61104076666b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@213b344a 2021-05-28T00:02:26.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc59b6ad-59eb-4c41-b988-61104076666b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc59b6ad-59eb-4c41-b988-61104076666b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b9e68a-7667-4043-b7ae-573ab076bd35 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b9e68a-7667-4043-b7ae-573ab076bd35 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6269f1c7 2021-05-28T00:02:26.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b9e68a-7667-4043-b7ae-573ab076bd35 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b9e68a-7667-4043-b7ae-573ab076bd35 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e554bdd4-48ab-4c04-b0ac-101529a1329a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e554bdd4-48ab-4c04-b0ac-101529a1329a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@391a6573 2021-05-28T00:02:26.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e554bdd4-48ab-4c04-b0ac-101529a1329a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e554bdd4-48ab-4c04-b0ac-101529a1329a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1963d7ea-3484-407e-b4c5-7405c5d14d15 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1963d7ea-3484-407e-b4c5-7405c5d14d15 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d8ad5ef 2021-05-28T00:02:26.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1963d7ea-3484-407e-b4c5-7405c5d14d15 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1963d7ea-3484-407e-b4c5-7405c5d14d15 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b51882-5d66-482f-9edf-a376e66a0264 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b51882-5d66-482f-9edf-a376e66a0264 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@756b2970 2021-05-28T00:02:26.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b51882-5d66-482f-9edf-a376e66a0264 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b51882-5d66-482f-9edf-a376e66a0264 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d412de88-c005-4d63-8a67-1e0be57df669 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d412de88-c005-4d63-8a67-1e0be57df669 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58a109d3 2021-05-28T00:02:26.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d412de88-c005-4d63-8a67-1e0be57df669 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.644Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d412de88-c005-4d63-8a67-1e0be57df669 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4856fdf2-376f-4f34-b68d-6d674aedead8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4856fdf2-376f-4f34-b68d-6d674aedead8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32face49 2021-05-28T00:02:26.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4856fdf2-376f-4f34-b68d-6d674aedead8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4856fdf2-376f-4f34-b68d-6d674aedead8 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e164725-16eb-46ec-9a4d-77401ff6ed23 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e164725-16eb-46ec-9a4d-77401ff6ed23 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69844054 2021-05-28T00:02:26.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e164725-16eb-46ec-9a4d-77401ff6ed23 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e164725-16eb-46ec-9a4d-77401ff6ed23 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b29c04-72e2-499b-b67f-bc50aed8382b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b29c04-72e2-499b-b67f-bc50aed8382b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4588c4bd 2021-05-28T00:02:26.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b29c04-72e2-499b-b67f-bc50aed8382b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37b29c04-72e2-499b-b67f-bc50aed8382b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838e6e12-7cf9-4fc2-b6a9-4b0ed8f30bbd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838e6e12-7cf9-4fc2-b6a9-4b0ed8f30bbd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1537b24e 2021-05-28T00:02:26.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838e6e12-7cf9-4fc2-b6a9-4b0ed8f30bbd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838e6e12-7cf9-4fc2-b6a9-4b0ed8f30bbd PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33859f85-5219-4fed-8f67-ebcd8a05e5d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33859f85-5219-4fed-8f67-ebcd8a05e5d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@715bbe27 2021-05-28T00:02:26.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33859f85-5219-4fed-8f67-ebcd8a05e5d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33859f85-5219-4fed-8f67-ebcd8a05e5d0 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=196f1217-bb76-4ff5-9cef-c506546b55be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=196f1217-bb76-4ff5-9cef-c506546b55be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a96c6b8 2021-05-28T00:02:26.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=196f1217-bb76-4ff5-9cef-c506546b55be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=196f1217-bb76-4ff5-9cef-c506546b55be PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95be4c99-0498-4dfe-b780-8f8ab32aa32d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95be4c99-0498-4dfe-b780-8f8ab32aa32d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3938507b 2021-05-28T00:02:26.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95be4c99-0498-4dfe-b780-8f8ab32aa32d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95be4c99-0498-4dfe-b780-8f8ab32aa32d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9db6f2be-66c3-4461-a0c0-6e1906995170 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9db6f2be-66c3-4461-a0c0-6e1906995170 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@113520d0 2021-05-28T00:02:26.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9db6f2be-66c3-4461-a0c0-6e1906995170 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9db6f2be-66c3-4461-a0c0-6e1906995170 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2831de1c-b559-4643-8ef2-a8d21682b1e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2831de1c-b559-4643-8ef2-a8d21682b1e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3db30e57 2021-05-28T00:02:26.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2831de1c-b559-4643-8ef2-a8d21682b1e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2831de1c-b559-4643-8ef2-a8d21682b1e7 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b23303-58f3-4b5d-a038-685ffd44c3ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b23303-58f3-4b5d-a038-685ffd44c3ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@311cac5f 2021-05-28T00:02:26.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b23303-58f3-4b5d-a038-685ffd44c3ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b23303-58f3-4b5d-a038-685ffd44c3ed PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d598b3-422b-46dc-98a3-aafd0ad45b58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d598b3-422b-46dc-98a3-aafd0ad45b58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4adc998f 2021-05-28T00:02:26.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d598b3-422b-46dc-98a3-aafd0ad45b58 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90d598b3-422b-46dc-98a3-aafd0ad45b58 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf1a2b99-162e-4abf-8ed0-7bd053c8af09 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf1a2b99-162e-4abf-8ed0-7bd053c8af09 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@725703a4 2021-05-28T00:02:26.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf1a2b99-162e-4abf-8ed0-7bd053c8af09 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf1a2b99-162e-4abf-8ed0-7bd053c8af09 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14d00530-3afe-409c-96c8-5e956a66f6bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14d00530-3afe-409c-96c8-5e956a66f6bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57336a0f 2021-05-28T00:02:26.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14d00530-3afe-409c-96c8-5e956a66f6bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14d00530-3afe-409c-96c8-5e956a66f6bf PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=853ba0fc-e388-45e4-afd9-7a94c4bc7723 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=853ba0fc-e388-45e4-afd9-7a94c4bc7723 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3994ae2 2021-05-28T00:02:26.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=853ba0fc-e388-45e4-afd9-7a94c4bc7723 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=853ba0fc-e388-45e4-afd9-7a94c4bc7723 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29fc8063-6bd9-4bd2-bee8-a8637a365e7e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29fc8063-6bd9-4bd2-bee8-a8637a365e7e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d27abff 2021-05-28T00:02:26.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29fc8063-6bd9-4bd2-bee8-a8637a365e7e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29fc8063-6bd9-4bd2-bee8-a8637a365e7e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ed6bba-9e6c-4b2c-b3da-691b1e172edd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ed6bba-9e6c-4b2c-b3da-691b1e172edd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6727c7dc 2021-05-28T00:02:26.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ed6bba-9e6c-4b2c-b3da-691b1e172edd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ed6bba-9e6c-4b2c-b3da-691b1e172edd PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a54f-8202-4aa2-8814-dee60ffbb99c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a54f-8202-4aa2-8814-dee60ffbb99c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b88ba25 2021-05-28T00:02:26.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a54f-8202-4aa2-8814-dee60ffbb99c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a54f-8202-4aa2-8814-dee60ffbb99c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf7562-c9ec-408d-be4f-9d49c477f6f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf7562-c9ec-408d-be4f-9d49c477f6f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@620f3bec 2021-05-28T00:02:26.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf7562-c9ec-408d-be4f-9d49c477f6f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf7562-c9ec-408d-be4f-9d49c477f6f7 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97be72df-1455-4770-9447-aeca2392ea3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97be72df-1455-4770-9447-aeca2392ea3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72670c01 2021-05-28T00:02:26.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97be72df-1455-4770-9447-aeca2392ea3f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97be72df-1455-4770-9447-aeca2392ea3f PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62a55d56-717a-4de7-8756-3331e7c97ea7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62a55d56-717a-4de7-8756-3331e7c97ea7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b272cb6 2021-05-28T00:02:26.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62a55d56-717a-4de7-8756-3331e7c97ea7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62a55d56-717a-4de7-8756-3331e7c97ea7 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741b8e99-74de-4e52-87f0-7e6c5cf8e1f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741b8e99-74de-4e52-87f0-7e6c5cf8e1f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4590f265 2021-05-28T00:02:26.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741b8e99-74de-4e52-87f0-7e6c5cf8e1f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741b8e99-74de-4e52-87f0-7e6c5cf8e1f9 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6754a8d-1d89-4fea-b2b7-730e96603150 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6754a8d-1d89-4fea-b2b7-730e96603150 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5edb7160 2021-05-28T00:02:26.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6754a8d-1d89-4fea-b2b7-730e96603150 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6754a8d-1d89-4fea-b2b7-730e96603150 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=689c7de2-47ae-4e03-bfdf-d0dac58717e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=689c7de2-47ae-4e03-bfdf-d0dac58717e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@141669be 2021-05-28T00:02:26.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=689c7de2-47ae-4e03-bfdf-d0dac58717e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=689c7de2-47ae-4e03-bfdf-d0dac58717e6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce428b31-2501-4ae0-8f5d-b3dc9e3e6cb4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce428b31-2501-4ae0-8f5d-b3dc9e3e6cb4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c6b516 2021-05-28T00:02:26.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce428b31-2501-4ae0-8f5d-b3dc9e3e6cb4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce428b31-2501-4ae0-8f5d-b3dc9e3e6cb4 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1417f991-faa1-4929-b966-ac6ee871ab47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1417f991-faa1-4929-b966-ac6ee871ab47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44077c71 2021-05-28T00:02:26.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1417f991-faa1-4929-b966-ac6ee871ab47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1417f991-faa1-4929-b966-ac6ee871ab47 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d28ca0-cc4c-4d37-969f-1692cd8ae9e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d28ca0-cc4c-4d37-969f-1692cd8ae9e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@415255e0 2021-05-28T00:02:26.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d28ca0-cc4c-4d37-969f-1692cd8ae9e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d28ca0-cc4c-4d37-969f-1692cd8ae9e5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=356b6bda-e531-4547-a28c-da31a8de3093 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=356b6bda-e531-4547-a28c-da31a8de3093 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66ae9fed 2021-05-28T00:02:26.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=356b6bda-e531-4547-a28c-da31a8de3093 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=356b6bda-e531-4547-a28c-da31a8de3093 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced5423e-cabf-4061-923e-cd1819f5a68e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced5423e-cabf-4061-923e-cd1819f5a68e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5573e82b 2021-05-28T00:02:26.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced5423e-cabf-4061-923e-cd1819f5a68e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ced5423e-cabf-4061-923e-cd1819f5a68e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.702Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b36d34e0-5072-40f6-a479-1f2ea98fa499 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.702Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b36d34e0-5072-40f6-a479-1f2ea98fa499 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ae8f200 2021-05-28T00:02:26.702Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b36d34e0-5072-40f6-a479-1f2ea98fa499 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.702Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b36d34e0-5072-40f6-a479-1f2ea98fa499 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0da9bca3-bb7e-4859-93a2-ed9883b2f6e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0da9bca3-bb7e-4859-93a2-ed9883b2f6e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79c65fe8 2021-05-28T00:02:26.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0da9bca3-bb7e-4859-93a2-ed9883b2f6e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0da9bca3-bb7e-4859-93a2-ed9883b2f6e5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.706Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e3983c3-bb8b-4809-b6d2-6668cfe7186c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.706Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e3983c3-bb8b-4809-b6d2-6668cfe7186c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b05e6ca 2021-05-28T00:02:26.706Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e3983c3-bb8b-4809-b6d2-6668cfe7186c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.706Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e3983c3-bb8b-4809-b6d2-6668cfe7186c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=580a4096-5080-493c-b486-afba72d1e950 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=580a4096-5080-493c-b486-afba72d1e950 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21fd6931 2021-05-28T00:02:26.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=580a4096-5080-493c-b486-afba72d1e950 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=580a4096-5080-493c-b486-afba72d1e950 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e7c02fb-ba0e-45f4-b4ef-b773a235b19c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e7c02fb-ba0e-45f4-b4ef-b773a235b19c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ce8f7fc 2021-05-28T00:02:26.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e7c02fb-ba0e-45f4-b4ef-b773a235b19c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e7c02fb-ba0e-45f4-b4ef-b773a235b19c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97c14a26-7e69-41f8-bf7e-3e206074a8c8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97c14a26-7e69-41f8-bf7e-3e206074a8c8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71ac896 2021-05-28T00:02:26.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97c14a26-7e69-41f8-bf7e-3e206074a8c8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97c14a26-7e69-41f8-bf7e-3e206074a8c8 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e206d90e-8d3e-4bb6-af74-17a37c19536b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e206d90e-8d3e-4bb6-af74-17a37c19536b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@436d4d09 2021-05-28T00:02:26.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e206d90e-8d3e-4bb6-af74-17a37c19536b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e206d90e-8d3e-4bb6-af74-17a37c19536b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d348cf-3db1-42f7-9482-d9977819b790 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d348cf-3db1-42f7-9482-d9977819b790 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@99310b4 2021-05-28T00:02:26.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d348cf-3db1-42f7-9482-d9977819b790 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d348cf-3db1-42f7-9482-d9977819b790 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba40ce93-e313-4bb9-bbb7-521c9e05b65f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba40ce93-e313-4bb9-bbb7-521c9e05b65f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24f516d 2021-05-28T00:02:26.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba40ce93-e313-4bb9-bbb7-521c9e05b65f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba40ce93-e313-4bb9-bbb7-521c9e05b65f PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.718Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bca9f1c-47ab-44c0-9d78-9b2d28a28cf5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.718Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bca9f1c-47ab-44c0-9d78-9b2d28a28cf5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53c067ca 2021-05-28T00:02:26.718Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bca9f1c-47ab-44c0-9d78-9b2d28a28cf5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.718Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bca9f1c-47ab-44c0-9d78-9b2d28a28cf5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.720Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68202148-77d6-431b-85bc-ec418fc7d3ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.720Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68202148-77d6-431b-85bc-ec418fc7d3ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7aec6dbe 2021-05-28T00:02:26.720Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68202148-77d6-431b-85bc-ec418fc7d3ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.720Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68202148-77d6-431b-85bc-ec418fc7d3ca PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=719f4bf3-d3db-42ec-b21d-c2c26e59de55 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=719f4bf3-d3db-42ec-b21d-c2c26e59de55 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b6bc4e7 2021-05-28T00:02:26.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=719f4bf3-d3db-42ec-b21d-c2c26e59de55 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=719f4bf3-d3db-42ec-b21d-c2c26e59de55 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.723Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4d7d969-5837-4858-bf6d-eca7110ba84a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.723Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4d7d969-5837-4858-bf6d-eca7110ba84a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@135724de 2021-05-28T00:02:26.723Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4d7d969-5837-4858-bf6d-eca7110ba84a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.723Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4d7d969-5837-4858-bf6d-eca7110ba84a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=457972d0-a2c2-4b49-84e2-5462488b108c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=457972d0-a2c2-4b49-84e2-5462488b108c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43f9ef49 2021-05-28T00:02:26.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=457972d0-a2c2-4b49-84e2-5462488b108c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=457972d0-a2c2-4b49-84e2-5462488b108c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0384306b-174b-43f1-bb99-08b05ab9de16 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0384306b-174b-43f1-bb99-08b05ab9de16 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a61fb15 2021-05-28T00:02:26.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0384306b-174b-43f1-bb99-08b05ab9de16 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0384306b-174b-43f1-bb99-08b05ab9de16 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.733Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9326f766-f225-4590-a7b6-eb06c56d1440 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.733Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9326f766-f225-4590-a7b6-eb06c56d1440 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f984042 2021-05-28T00:02:26.733Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9326f766-f225-4590-a7b6-eb06c56d1440 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.733Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9326f766-f225-4590-a7b6-eb06c56d1440 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.734Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=410dec2a-dae1-49ef-ad93-cc13973e169d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.734Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=410dec2a-dae1-49ef-ad93-cc13973e169d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4543f8bb 2021-05-28T00:02:26.734Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=410dec2a-dae1-49ef-ad93-cc13973e169d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.734Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=410dec2a-dae1-49ef-ad93-cc13973e169d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=048da313-0de4-4729-a1df-5c8e88d685d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=048da313-0de4-4729-a1df-5c8e88d685d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@494cd4fd 2021-05-28T00:02:26.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=048da313-0de4-4729-a1df-5c8e88d685d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.737Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=048da313-0de4-4729-a1df-5c8e88d685d3 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.739Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ae60b7-b0e4-4b51-b785-a4ac5a26dd26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.739Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ae60b7-b0e4-4b51-b785-a4ac5a26dd26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ca02792 2021-05-28T00:02:26.739Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ae60b7-b0e4-4b51-b785-a4ac5a26dd26 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.739Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ae60b7-b0e4-4b51-b785-a4ac5a26dd26 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5557b917-dd15-44e6-858a-4daa8c214f1a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5557b917-dd15-44e6-858a-4daa8c214f1a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3117fe14 2021-05-28T00:02:26.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5557b917-dd15-44e6-858a-4daa8c214f1a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.741Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5557b917-dd15-44e6-858a-4daa8c214f1a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.743Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538698fb-14fe-428e-9fb4-a15d23456696 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.743Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538698fb-14fe-428e-9fb4-a15d23456696 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bcb4ead 2021-05-28T00:02:26.743Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538698fb-14fe-428e-9fb4-a15d23456696 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.743Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538698fb-14fe-428e-9fb4-a15d23456696 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8e4d5-d8af-43e3-996e-d8c3de0c1725 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8e4d5-d8af-43e3-996e-d8c3de0c1725 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@448f8415 2021-05-28T00:02:26.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8e4d5-d8af-43e3-996e-d8c3de0c1725 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8e4d5-d8af-43e3-996e-d8c3de0c1725 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd042279-ad50-433f-9e4b-61b702956b3e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd042279-ad50-433f-9e4b-61b702956b3e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@598ea7a1 2021-05-28T00:02:26.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd042279-ad50-433f-9e4b-61b702956b3e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd042279-ad50-433f-9e4b-61b702956b3e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5a826d-ffbb-474a-877b-747097b1de59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5a826d-ffbb-474a-877b-747097b1de59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@576705da 2021-05-28T00:02:26.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5a826d-ffbb-474a-877b-747097b1de59 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5a826d-ffbb-474a-877b-747097b1de59 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828a3c16-a1fe-41d5-80b9-3012d7f86c82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828a3c16-a1fe-41d5-80b9-3012d7f86c82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@666887bd 2021-05-28T00:02:26.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828a3c16-a1fe-41d5-80b9-3012d7f86c82 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828a3c16-a1fe-41d5-80b9-3012d7f86c82 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.753Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=020a9110-6596-4c57-8a19-09f35cb94590 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.753Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=020a9110-6596-4c57-8a19-09f35cb94590 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5eff0849 2021-05-28T00:02:26.753Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=020a9110-6596-4c57-8a19-09f35cb94590 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.753Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=020a9110-6596-4c57-8a19-09f35cb94590 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d650ed8-3527-4742-bfc4-cdcd39f8021c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d650ed8-3527-4742-bfc4-cdcd39f8021c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72e01f98 2021-05-28T00:02:26.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d650ed8-3527-4742-bfc4-cdcd39f8021c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d650ed8-3527-4742-bfc4-cdcd39f8021c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b69e959-7f32-406e-804a-0532407bf467 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b69e959-7f32-406e-804a-0532407bf467 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d469dcb 2021-05-28T00:02:26.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b69e959-7f32-406e-804a-0532407bf467 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b69e959-7f32-406e-804a-0532407bf467 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41881600-8fa2-444e-8b04-9aca9b06f682 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41881600-8fa2-444e-8b04-9aca9b06f682 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f31a4b1 2021-05-28T00:02:26.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41881600-8fa2-444e-8b04-9aca9b06f682 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41881600-8fa2-444e-8b04-9aca9b06f682 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d0fcf6-629a-4b0e-8ae6-fffca0925429 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d0fcf6-629a-4b0e-8ae6-fffca0925429 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@192a5e5e 2021-05-28T00:02:26.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d0fcf6-629a-4b0e-8ae6-fffca0925429 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d0fcf6-629a-4b0e-8ae6-fffca0925429 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e708138c-e7d3-4fba-9866-61bb0dc97e5d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e708138c-e7d3-4fba-9866-61bb0dc97e5d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bc48e3c 2021-05-28T00:02:26.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e708138c-e7d3-4fba-9866-61bb0dc97e5d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e708138c-e7d3-4fba-9866-61bb0dc97e5d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c8223c8-381b-4e16-b081-bfeab584ad81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c8223c8-381b-4e16-b081-bfeab584ad81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4245ee4a 2021-05-28T00:02:26.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c8223c8-381b-4e16-b081-bfeab584ad81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c8223c8-381b-4e16-b081-bfeab584ad81 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:26.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e77eabe-d59d-4887-ae95-789912d2cb82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:26.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e77eabe-d59d-4887-ae95-789912d2cb82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70e3de9f 2021-05-28T00:02:26.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e77eabe-d59d-4887-ae95-789912d2cb82 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:26.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e77eabe-d59d-4887-ae95-789912d2cb82 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:27.255Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:02:28.267Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1698 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:26.569Z InvocationID=0775d6fc-6a50-44cb-aeca-713ed4d5462c LogTimestamp=2021-05-28T00:02:28.267239Z ResponseDesc=Created Exiting. 2021-05-28T00:02:28.267Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:28.281Z [qtp580718781-106] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:28.281037Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:28.281Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:28.281037Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:28.281Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:02:28.377Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4b845aab-e619-45b8-90b0-c48ec3323214 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:28.377410Z InvocationID=b59cb9f4-a8f4-4496-a765-b3dda3fb4ac9 LogTimestamp=2021-05-28T00:02:28.377449Z Entering 2021-05-28T00:02:28.377Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4b845aab-e619-45b8-90b0-c48ec3323214 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:28.377410Z InvocationID=b59cb9f4-a8f4-4496-a765-b3dda3fb4ac9 LogTimestamp=2021-05-28T00:02:28.377449Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:28.473Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:28.377Z InvocationID=366920aa-1861-467a-9a49-5e4d636b04db PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:02:29.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17235420-768e-42d0-98d9-17a8f006d27c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:29.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17235420-768e-42d0-98d9-17a8f006d27c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bf0b673 2021-05-28T00:02:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17235420-768e-42d0-98d9-17a8f006d27c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=17235420-768e-42d0-98d9-17a8f006d27c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:29.773Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:02:29.774Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 128000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:02:29.870Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:02:29.871Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 128000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:02:30.468Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2091 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:28.377Z InvocationID=366920aa-1861-467a-9a49-5e4d636b04db LogTimestamp=2021-05-28T00:02:30.468281Z ResponseDesc=Created Exiting. 2021-05-28T00:02:30.468Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:30.576Z [qtp580718781-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:30.576466Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:30.576Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:30.576466Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:30.576Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:02:30.671Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8bf15614-a6d2-42ca-8e53-e7c728ce3f18 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:30.671434Z InvocationID=64370616-8585-4d22-ae65-754530a0c241 LogTimestamp=2021-05-28T00:02:30.671470Z Entering 2021-05-28T00:02:30.671Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8bf15614-a6d2-42ca-8e53-e7c728ce3f18 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:30.671434Z InvocationID=64370616-8585-4d22-ae65-754530a0c241 LogTimestamp=2021-05-28T00:02:30.671470Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:30.771Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:30.671Z InvocationID=778171c8-cf6c-418c-829b-c49a7262b471 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:02:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d98994ce-d677-476d-af5a-b06058052ea4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d98994ce-d677-476d-af5a-b06058052ea4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@740a6a5f 2021-05-28T00:02:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d98994ce-d677-476d-af5a-b06058052ea4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d98994ce-d677-476d-af5a-b06058052ea4 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:32.968Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2296 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:30.671Z InvocationID=778171c8-cf6c-418c-829b-c49a7262b471 LogTimestamp=2021-05-28T00:02:32.967727Z ResponseDesc=Created Exiting. 2021-05-28T00:02:32.968Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:33.170Z [qtp580718781-102] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:33.170753Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:33.170Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:33.170753Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:33.171Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:02:33.369Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8f7d7188-218f-4be0-9a66-61e0bc332a21 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:33.369627Z InvocationID=123cf20f-bced-4ff1-9fc0-80c5c6310c92 LogTimestamp=2021-05-28T00:02:33.369693Z Entering 2021-05-28T00:02:33.370Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8f7d7188-218f-4be0-9a66-61e0bc332a21 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:33.369627Z InvocationID=123cf20f-bced-4ff1-9fc0-80c5c6310c92 LogTimestamp=2021-05-28T00:02:33.369693Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:33.470Z [qtp580718781-107] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:33.370Z InvocationID=ecf5b925-225e-49a6-a671-72c094362a8d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:02:35.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b66f815-8783-4fc2-86d9-62d0df4a3d75 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b66f815-8783-4fc2-86d9-62d0df4a3d75 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1950ed2a 2021-05-28T00:02:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b66f815-8783-4fc2-86d9-62d0df4a3d75 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b66f815-8783-4fc2-86d9-62d0df4a3d75 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:36.157Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2787 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:33.370Z InvocationID=ecf5b925-225e-49a6-a671-72c094362a8d LogTimestamp=2021-05-28T00:02:36.157498Z ResponseDesc=Created Exiting. 2021-05-28T00:02:36.158Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:36.168Z [qtp580718781-100] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:36.168555Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:36.168Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:36.168555Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:36.169Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:02:36.259Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=678bf2a7-d4c1-4ca2-93a8-cbc268b5ebc6 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:36.259617Z InvocationID=4818cc05-ca7d-4719-acd2-5f27f3e6de9f LogTimestamp=2021-05-28T00:02:36.259681Z Entering 2021-05-28T00:02:36.260Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=678bf2a7-d4c1-4ca2-93a8-cbc268b5ebc6 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:36.259617Z InvocationID=4818cc05-ca7d-4719-acd2-5f27f3e6de9f LogTimestamp=2021-05-28T00:02:36.259681Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:36.275Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:36.260Z InvocationID=b769280f-d125-4ef5-bc81-ee6a46a7e231 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:02:36.305Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 32000 milliseconds 2021-05-28T00:02:37.976Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1716 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:36.260Z InvocationID=b769280f-d125-4ef5-bc81-ee6a46a7e231 LogTimestamp=2021-05-28T00:02:37.976396Z ResponseDesc=Created Exiting. 2021-05-28T00:02:37.976Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:38.076Z [qtp580718781-102] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:38.076123Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:38.076Z [qtp580718781-102] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:38.076123Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:38.076Z [qtp580718781-102] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:02:38.173Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=405c2dd5-0800-4bdf-90ba-ea161efdd9e4 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:38.173701Z InvocationID=e06892db-dfca-48db-b935-5f35b61103ee LogTimestamp=2021-05-28T00:02:38.173751Z Entering 2021-05-28T00:02:38.174Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=405c2dd5-0800-4bdf-90ba-ea161efdd9e4 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:38.173701Z InvocationID=e06892db-dfca-48db-b935-5f35b61103ee LogTimestamp=2021-05-28T00:02:38.173751Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:38.182Z [qtp580718781-107] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:38.174Z InvocationID=0c68e402-0f78-409f-9519-c4976925712e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:02:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a80fa7b-649f-4205-9ea7-0d1ec1732ab1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a80fa7b-649f-4205-9ea7-0d1ec1732ab1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77a9dfb9 2021-05-28T00:02:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a80fa7b-649f-4205-9ea7-0d1ec1732ab1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a80fa7b-649f-4205-9ea7-0d1ec1732ab1 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:39.973Z [qtp580718781-107] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1798 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:38.174Z InvocationID=0c68e402-0f78-409f-9519-c4976925712e LogTimestamp=2021-05-28T00:02:39.972688Z ResponseDesc=Created Exiting. 2021-05-28T00:02:39.973Z [qtp580718781-107] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:39.985Z [qtp580718781-108] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:39.984940Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:39.985Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:39.984940Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:39.985Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:02:40.084Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ddfd6099-2724-4161-81fc-265b61bea291 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:40.083888Z InvocationID=ef455c5a-62c5-4ef2-bb78-dd22ebb93ad4 LogTimestamp=2021-05-28T00:02:40.083953Z Entering 2021-05-28T00:02:40.084Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ddfd6099-2724-4161-81fc-265b61bea291 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:40.083888Z InvocationID=ef455c5a-62c5-4ef2-bb78-dd22ebb93ad4 LogTimestamp=2021-05-28T00:02:40.083953Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:40.171Z [qtp580718781-106] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:40.084Z InvocationID=26e06691-fbdf-49db-85c4-987eafc2b3d5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:02:40.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:02:40.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:02:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2923f008-efaa-476a-860a-e7e9b10c7937 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2923f008-efaa-476a-860a-e7e9b10c7937 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4954ac11 2021-05-28T00:02:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2923f008-efaa-476a-860a-e7e9b10c7937 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2923f008-efaa-476a-860a-e7e9b10c7937 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:43.074Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2990 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:40.084Z InvocationID=26e06691-fbdf-49db-85c4-987eafc2b3d5 LogTimestamp=2021-05-28T00:02:43.074670Z ResponseDesc=Created Exiting. 2021-05-28T00:02:43.075Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:43.269Z [qtp580718781-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:43.268984Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:43.269Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:43.268984Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:43.269Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:02:43.373Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7960df65-2862-4171-9578-3cf27d190878 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:43.373331Z InvocationID=7bfafac7-b380-4482-a977-3b052e63074e LogTimestamp=2021-05-28T00:02:43.373377Z Entering 2021-05-28T00:02:43.373Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7960df65-2862-4171-9578-3cf27d190878 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:43.373331Z InvocationID=7bfafac7-b380-4482-a977-3b052e63074e LogTimestamp=2021-05-28T00:02:43.373377Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:43.656Z [qtp580718781-100] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:43.373Z InvocationID=51ea59fd-5502-4dc7-abd4-d825cda2c46f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:02:43.875Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:02:44.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdb9eaf5-7fb2-45e7-bc98-973d8387289b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:44.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdb9eaf5-7fb2-45e7-bc98-973d8387289b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e2b6ff6 2021-05-28T00:02:44.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdb9eaf5-7fb2-45e7-bc98-973d8387289b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:44.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdb9eaf5-7fb2-45e7-bc98-973d8387289b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:46.372Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2998 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:43.373Z InvocationID=51ea59fd-5502-4dc7-abd4-d825cda2c46f LogTimestamp=2021-05-28T00:02:46.371688Z ResponseDesc=Created Exiting. 2021-05-28T00:02:46.372Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:46.464Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:46.464386Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:02:46.464Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:02:46.464386Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:46.465Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:02:46.560Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4cc8cd40-6e71-4fed-a191-6defe9ae7287 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:46.560658Z InvocationID=8fb9c845-3f3e-49ed-92c3-349c8ea9e55f LogTimestamp=2021-05-28T00:02:46.560702Z Entering 2021-05-28T00:02:46.560Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4cc8cd40-6e71-4fed-a191-6defe9ae7287 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:46.560658Z InvocationID=8fb9c845-3f3e-49ed-92c3-349c8ea9e55f LogTimestamp=2021-05-28T00:02:46.560702Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:02:46.568Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:46.561Z InvocationID=56cdccab-248a-4de2-b05f-e65c2df82a77 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:02:47.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e75f222-e3a3-4471-ac59-41babd910731 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:47.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e75f222-e3a3-4471-ac59-41babd910731 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@708ce6a4 2021-05-28T00:02:47.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e75f222-e3a3-4471-ac59-41babd910731 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:47.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e75f222-e3a3-4471-ac59-41babd910731 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:48.081Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1519 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:02:46.561Z InvocationID=56cdccab-248a-4de2-b05f-e65c2df82a77 LogTimestamp=2021-05-28T00:02:48.080967Z ResponseDesc=Created Exiting. 2021-05-28T00:02:48.081Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:02:48.481Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=79e1351f-5e60-4a98-9d5b-0d4791c7400e InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:48.481350Z InvocationID=5d532a4a-eb9d-48b2-a990-e9d25627ad17 LogTimestamp=2021-05-28T00:02:48.481420Z Entering 2021-05-28T00:02:48.482Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=79e1351f-5e60-4a98-9d5b-0d4791c7400e InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:48.481350Z InvocationID=5d532a4a-eb9d-48b2-a990-e9d25627ad17 LogTimestamp=2021-05-28T00:02:48.481420Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:02:48.569Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=87 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:02:48.482Z InvocationID=d3ab99e8-5221-46f8-b1c1-2f2e93354bf5 LogTimestamp=2021-05-28T00:02:48.569196Z ResponseDesc=OK Exiting. 2021-05-28T00:02:48.569Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:02:49.077Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ccd6ba04-0ad5-480a-ad5c-ef8363c72592 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:49.077077Z InvocationID=c334d51e-4846-499b-a538-cb807a2498d7 LogTimestamp=2021-05-28T00:02:49.077140Z Entering 2021-05-28T00:02:49.077Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ccd6ba04-0ad5-480a-ad5c-ef8363c72592 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:49.077077Z InvocationID=c334d51e-4846-499b-a538-cb807a2498d7 LogTimestamp=2021-05-28T00:02:49.077140Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:02:49.272Z [qtp580718781-108] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:02:49.077Z InvocationID=62f28776-ad03-4479-9c38-5fe8bd5cb7fb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 192817596 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n192817596 100% \n 2021-05-28T00:02:49.281Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=204 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:02:49.077Z InvocationID=62f28776-ad03-4479-9c38-5fe8bd5cb7fb LogTimestamp=2021-05-28T00:02:49.281175Z ResponseDesc=OK Exiting. 2021-05-28T00:02:49.281Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:02:49.378Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=92915e51-05f6-48e0-993e-b3e5ed59c376 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:49.378374Z InvocationID=93b5e995-07d9-4681-b698-f0a4ada806c7 LogTimestamp=2021-05-28T00:02:49.378415Z Entering 2021-05-28T00:02:49.378Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=92915e51-05f6-48e0-993e-b3e5ed59c376 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:49.378374Z InvocationID=93b5e995-07d9-4681-b698-f0a4ada806c7 LogTimestamp=2021-05-28T00:02:49.378415Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:02:49.484Z [qtp580718781-106] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:02:49.378Z InvocationID=8d2da0a7-d9e7-42a8-8c19-3d92fceacdea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:02:49.574Z [qtp580718781-106] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:02:49.378Z InvocationID=8d2da0a7-d9e7-42a8-8c19-3d92fceacdea ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:02:49.574Z [qtp580718781-106] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:02:49.378Z InvocationID=8d2da0a7-d9e7-42a8-8c19-3d92fceacdea ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:02:49.575Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=197 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:02:49.378Z InvocationID=8d2da0a7-d9e7-42a8-8c19-3d92fceacdea ErrorCode=500 LogTimestamp=2021-05-28T00:02:49.575212Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:49.575Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:02:50.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=269b00b6-a57d-4083-bbfe-85f52e0b6a2c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:50.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=269b00b6-a57d-4083-bbfe-85f52e0b6a2c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4961bb1b 2021-05-28T00:02:50.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=269b00b6-a57d-4083-bbfe-85f52e0b6a2c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:50.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=269b00b6-a57d-4083-bbfe-85f52e0b6a2c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09c74624-2a41-4005-8821-ea99f6622c17 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09c74624-2a41-4005-8821-ea99f6622c17 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25b9a202 2021-05-28T00:02:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09c74624-2a41-4005-8821-ea99f6622c17 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09c74624-2a41-4005-8821-ea99f6622c17 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:55.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:02:55.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:02:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fc7c65b-b43e-4db6-9191-380f7ea950f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fc7c65b-b43e-4db6-9191-380f7ea950f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10d83d9c 2021-05-28T00:02:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fc7c65b-b43e-4db6-9191-380f7ea950f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fc7c65b-b43e-4db6-9191-380f7ea950f8 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:59.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8527ceee-7471-4c97-b110-d2e901e6c9eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:02:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8527ceee-7471-4c97-b110-d2e901e6c9eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f650e5 2021-05-28T00:02:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8527ceee-7471-4c97-b110-d2e901e6c9eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:02:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8527ceee-7471-4c97-b110-d2e901e6c9eb PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:02:59.759Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cdfcb410-1ddd-4a94-a260-cd754c44259e InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:59.759564Z InvocationID=ab1555b1-fce6-4f88-b493-5ee436d82c71 LogTimestamp=2021-05-28T00:02:59.759663Z Entering 2021-05-28T00:02:59.760Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cdfcb410-1ddd-4a94-a260-cd754c44259e InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:02:59.759564Z InvocationID=ab1555b1-fce6-4f88-b493-5ee436d82c71 LogTimestamp=2021-05-28T00:02:59.759663Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:02:59.874Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:02:59.760Z InvocationID=c2481569-e21f-4524-af2a-7f7eee84f169 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:02:59.882Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:02:59.760Z InvocationID=c2481569-e21f-4524-af2a-7f7eee84f169 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:02:59.882Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:02:59.760Z InvocationID=c2481569-e21f-4524-af2a-7f7eee84f169 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:02:59.883Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=123 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:02:59.760Z InvocationID=c2481569-e21f-4524-af2a-7f7eee84f169 ErrorCode=500 LogTimestamp=2021-05-28T00:02:59.883015Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:02:59.883Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:03:02.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f2e8e45-2428-41d0-b2a5-f46bd99fc562 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f2e8e45-2428-41d0-b2a5-f46bd99fc562 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@430b8a63 2021-05-28T00:03:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f2e8e45-2428-41d0-b2a5-f46bd99fc562 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f2e8e45-2428-41d0-b2a5-f46bd99fc562 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:05.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7311d76a-4f9c-4031-a843-d22130f6cfb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:05.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7311d76a-4f9c-4031-a843-d22130f6cfb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bca99b2 2021-05-28T00:03:05.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7311d76a-4f9c-4031-a843-d22130f6cfb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:05.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7311d76a-4f9c-4031-a843-d22130f6cfb9 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:08.306Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 64000 milliseconds 2021-05-28T00:03:08.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d576ed7f-0325-4651-b224-4a985e84ac2e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:08.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d576ed7f-0325-4651-b224-4a985e84ac2e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7aaf758a 2021-05-28T00:03:08.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d576ed7f-0325-4651-b224-4a985e84ac2e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:08.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d576ed7f-0325-4651-b224-4a985e84ac2e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:09.985Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1d13f368-1943-4d7e-9002-17d93025c897 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:09.985668Z InvocationID=e2aab147-01fc-4bde-b710-e9fd1722d34a LogTimestamp=2021-05-28T00:03:09.985734Z Entering 2021-05-28T00:03:09.986Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1d13f368-1943-4d7e-9002-17d93025c897 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:09.985668Z InvocationID=e2aab147-01fc-4bde-b710-e9fd1722d34a LogTimestamp=2021-05-28T00:03:09.985734Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:03:10.458Z [qtp580718781-100] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:03:09.986Z InvocationID=d0eeae80-64b7-4e24-8d2c-6a8c34611709 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:03:10.472Z [qtp580718781-100] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:09.986Z InvocationID=d0eeae80-64b7-4e24-8d2c-6a8c34611709 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:03:10.472Z [qtp580718781-100] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:09.986Z InvocationID=d0eeae80-64b7-4e24-8d2c-6a8c34611709 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:03:10.473Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=487 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:03:09.986Z InvocationID=d0eeae80-64b7-4e24-8d2c-6a8c34611709 ErrorCode=500 LogTimestamp=2021-05-28T00:03:10.473244Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:03:10.473Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:03:10.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:03:10.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:03:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04c59e94-59dd-4a65-b9ac-5e525d3c535d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04c59e94-59dd-4a65-b9ac-5e525d3c535d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f6b7342 2021-05-28T00:03:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04c59e94-59dd-4a65-b9ac-5e525d3c535d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04c59e94-59dd-4a65-b9ac-5e525d3c535d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:12.292Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:03:12.293Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:03:12.293Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:03:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d94185-fdc8-4343-986d-11e5556e06a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d94185-fdc8-4343-986d-11e5556e06a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@502ca05e 2021-05-28T00:03:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d94185-fdc8-4343-986d-11e5556e06a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:14.734Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d94185-fdc8-4343-986d-11e5556e06a1 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:17.469Z [cluster2-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:03:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ddef23-e74b-4601-9531-adbafe6698b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ddef23-e74b-4601-9531-adbafe6698b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f54b4e0 2021-05-28T00:03:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ddef23-e74b-4601-9531-adbafe6698b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ddef23-e74b-4601-9531-adbafe6698b2 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:20.581Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d85cc945-94d7-473e-b7e1-4efbdf28de5a InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:20.580973Z InvocationID=590b1ee3-037a-48cb-9663-4e58a44bb3d5 LogTimestamp=2021-05-28T00:03:20.581056Z Entering 2021-05-28T00:03:20.581Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d85cc945-94d7-473e-b7e1-4efbdf28de5a InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:20.580973Z InvocationID=590b1ee3-037a-48cb-9663-4e58a44bb3d5 LogTimestamp=2021-05-28T00:03:20.581056Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:03:20.690Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:03:20.581Z InvocationID=8965f40c-d933-4e6a-be22-76ea428d2468 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:03:20.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b1ec38-9495-4441-bdb6-5cc090412fc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b1ec38-9495-4441-bdb6-5cc090412fc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e5687a5 2021-05-28T00:03:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b1ec38-9495-4441-bdb6-5cc090412fc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b1ec38-9495-4441-bdb6-5cc090412fc1 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:20.868Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:20.581Z InvocationID=8965f40c-d933-4e6a-be22-76ea428d2468 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:03:20.868Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:20.581Z InvocationID=8965f40c-d933-4e6a-be22-76ea428d2468 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:03:20.869Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=288 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:03:20.581Z InvocationID=8965f40c-d933-4e6a-be22-76ea428d2468 ErrorCode=500 LogTimestamp=2021-05-28T00:03:20.869294Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:03:20.869Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:03:23.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6af0ddb-d1d5-416e-b743-95c92ff51a29 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:23.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6af0ddb-d1d5-416e-b743-95c92ff51a29 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a33004e 2021-05-28T00:03:23.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6af0ddb-d1d5-416e-b743-95c92ff51a29 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:23.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6af0ddb-d1d5-416e-b743-95c92ff51a29 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:25.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:03:25.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:03:26.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ec100c-a745-454a-ba28-a64d4a79f660 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:26.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ec100c-a745-454a-ba28-a64d4a79f660 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fa87906 2021-05-28T00:03:26.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ec100c-a745-454a-ba28-a64d4a79f660 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:26.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ec100c-a745-454a-ba28-a64d4a79f660 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:27.375Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:03:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1342f116-76d8-4f7e-bd77-21f9facccd8d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1342f116-76d8-4f7e-bd77-21f9facccd8d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5512fa7 2021-05-28T00:03:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1342f116-76d8-4f7e-bd77-21f9facccd8d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1342f116-76d8-4f7e-bd77-21f9facccd8d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:30.977Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=da2c03c7-6345-4520-b4b2-36e75fe741d4 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:30.977374Z InvocationID=d1ea655b-2751-45e7-ad16-eff0d58ddf92 LogTimestamp=2021-05-28T00:03:30.977435Z Entering 2021-05-28T00:03:30.977Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=da2c03c7-6345-4520-b4b2-36e75fe741d4 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:30.977374Z InvocationID=d1ea655b-2751-45e7-ad16-eff0d58ddf92 LogTimestamp=2021-05-28T00:03:30.977435Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:03:31.087Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:03:30.978Z InvocationID=7309e3c0-e735-4cb8-ba48-46ad04f378f4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:03:31.268Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:30.978Z InvocationID=7309e3c0-e735-4cb8-ba48-46ad04f378f4 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:03:31.268Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:30.978Z InvocationID=7309e3c0-e735-4cb8-ba48-46ad04f378f4 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:03:31.269Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=290 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:03:30.978Z InvocationID=7309e3c0-e735-4cb8-ba48-46ad04f378f4 ErrorCode=500 LogTimestamp=2021-05-28T00:03:31.268937Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:03:31.269Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:03:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbbc7084-8335-47cf-a94b-c00ed6ab373d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbbc7084-8335-47cf-a94b-c00ed6ab373d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f20fb07 2021-05-28T00:03:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbbc7084-8335-47cf-a94b-c00ed6ab373d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbbc7084-8335-47cf-a94b-c00ed6ab373d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:35.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e85afc6-dfe2-46d9-9079-637852d6745c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e85afc6-dfe2-46d9-9079-637852d6745c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3db3a9dd 2021-05-28T00:03:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e85afc6-dfe2-46d9-9079-637852d6745c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e85afc6-dfe2-46d9-9079-637852d6745c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:38.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c758fc58-fcef-4e78-9a0d-878cd869499f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:38.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c758fc58-fcef-4e78-9a0d-878cd869499f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@711f8724 2021-05-28T00:03:38.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c758fc58-fcef-4e78-9a0d-878cd869499f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:38.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c758fc58-fcef-4e78-9a0d-878cd869499f PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:40.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:03:40.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:03:41.486Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0dfb64cb-fd73-49bf-92ed-396b079765e5 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:41.486268Z InvocationID=e6eca378-f48f-41ff-8fa3-9a689980d88f LogTimestamp=2021-05-28T00:03:41.486332Z Entering 2021-05-28T00:03:41.486Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0dfb64cb-fd73-49bf-92ed-396b079765e5 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:41.486268Z InvocationID=e6eca378-f48f-41ff-8fa3-9a689980d88f LogTimestamp=2021-05-28T00:03:41.486332Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:03:41.589Z [qtp580718781-106] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:03:41.486Z InvocationID=68428658-f8c6-4447-a1b7-a127d19125a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:03:41.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217cc765-ce9d-4b91-a52d-40ef2df0bbe6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:41.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217cc765-ce9d-4b91-a52d-40ef2df0bbe6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d3fa2b7 2021-05-28T00:03:41.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217cc765-ce9d-4b91-a52d-40ef2df0bbe6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217cc765-ce9d-4b91-a52d-40ef2df0bbe6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:41.767Z [qtp580718781-106] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:41.486Z InvocationID=68428658-f8c6-4447-a1b7-a127d19125a6 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:03:41.768Z [qtp580718781-106] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:41.486Z InvocationID=68428658-f8c6-4447-a1b7-a127d19125a6 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:03:41.768Z [qtp580718781-106] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=282 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:03:41.486Z InvocationID=68428658-f8c6-4447-a1b7-a127d19125a6 ErrorCode=500 LogTimestamp=2021-05-28T00:03:41.768618Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:03:41.769Z [qtp580718781-106] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:03:44.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03b8e2b9-4310-4e52-9d77-9f38add9fc2a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:44.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03b8e2b9-4310-4e52-9d77-9f38add9fc2a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@655f86d4 2021-05-28T00:03:44.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03b8e2b9-4310-4e52-9d77-9f38add9fc2a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:44.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03b8e2b9-4310-4e52-9d77-9f38add9fc2a PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:47.478Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:03:47.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53d0a9c6-aafa-49e8-9e0d-40b55b1f6d19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:47.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53d0a9c6-aafa-49e8-9e0d-40b55b1f6d19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@450236e3 2021-05-28T00:03:47.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53d0a9c6-aafa-49e8-9e0d-40b55b1f6d19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:47.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53d0a9c6-aafa-49e8-9e0d-40b55b1f6d19 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:50.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02cf86b7-ce32-473f-a91f-468ca29db188 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:50.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02cf86b7-ce32-473f-a91f-468ca29db188 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4829afc0 2021-05-28T00:03:50.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02cf86b7-ce32-473f-a91f-468ca29db188 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:50.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02cf86b7-ce32-473f-a91f-468ca29db188 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:51.873Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0b134fe3-c413-44c9-a94a-48f9bf9c572d InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:51.873104Z InvocationID=96c3cfd3-afb8-426b-b737-339ff1938bba LogTimestamp=2021-05-28T00:03:51.873196Z Entering 2021-05-28T00:03:51.873Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0b134fe3-c413-44c9-a94a-48f9bf9c572d InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:03:51.873104Z InvocationID=96c3cfd3-afb8-426b-b737-339ff1938bba LogTimestamp=2021-05-28T00:03:51.873196Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:03:51.971Z [qtp580718781-108] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:03:51.874Z InvocationID=09ddc613-b573-445f-8cec-5acadab8271e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:03:52.178Z [qtp580718781-108] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:51.874Z InvocationID=09ddc613-b573-445f-8cec-5acadab8271e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:03:52.179Z [qtp580718781-108] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:03:51.874Z InvocationID=09ddc613-b573-445f-8cec-5acadab8271e ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:03:52.180Z [qtp580718781-108] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=306 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:03:51.874Z InvocationID=09ddc613-b573-445f-8cec-5acadab8271e ErrorCode=500 LogTimestamp=2021-05-28T00:03:52.180157Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:03:52.180Z [qtp580718781-108] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:03:53.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ca27da2-b31e-4669-b772-4200966e2479 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ca27da2-b31e-4669-b772-4200966e2479 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48038668 2021-05-28T00:03:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ca27da2-b31e-4669-b772-4200966e2479 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ca27da2-b31e-4669-b772-4200966e2479 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:55.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:03:55.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:03:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a36965-dc0d-4328-8ca5-37eca6f1f7a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a36965-dc0d-4328-8ca5-37eca6f1f7a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51f3d09a 2021-05-28T00:03:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a36965-dc0d-4328-8ca5-37eca6f1f7a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a36965-dc0d-4328-8ca5-37eca6f1f7a8 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:03:59.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a34075-8098-49b8-b46c-d212fb686e1c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:03:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a34075-8098-49b8-b46c-d212fb686e1c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5924169 2021-05-28T00:03:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a34075-8098-49b8-b46c-d212fb686e1c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:03:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7a34075-8098-49b8-b46c-d212fb686e1c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:02.376Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b0cdd2ea-f036-41a9-a75c-a9dff63aa5ce InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:02.375929Z InvocationID=19a14248-eb08-4409-a2d4-696f394ff773 LogTimestamp=2021-05-28T00:04:02.376009Z Entering 2021-05-28T00:04:02.376Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b0cdd2ea-f036-41a9-a75c-a9dff63aa5ce InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:02.375929Z InvocationID=19a14248-eb08-4409-a2d4-696f394ff773 LogTimestamp=2021-05-28T00:04:02.376009Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:04:02.471Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:04:02.376Z InvocationID=061b7fa7-2750-4e7c-bb87-43ae28604c5d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:04:02.571Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:02.376Z InvocationID=061b7fa7-2750-4e7c-bb87-43ae28604c5d ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:04:02.572Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:02.376Z InvocationID=061b7fa7-2750-4e7c-bb87-43ae28604c5d ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:04:02.573Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=196 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:04:02.376Z InvocationID=061b7fa7-2750-4e7c-bb87-43ae28604c5d ErrorCode=500 LogTimestamp=2021-05-28T00:04:02.572789Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:04:02.573Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:04:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bee951fc-8614-461e-a010-69ee3f38e48b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bee951fc-8614-461e-a010-69ee3f38e48b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40a9b4 2021-05-28T00:04:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bee951fc-8614-461e-a010-69ee3f38e48b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bee951fc-8614-461e-a010-69ee3f38e48b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:05.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e134bb-e0a6-4e3b-95ad-c7c2aec150ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:05.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e134bb-e0a6-4e3b-95ad-c7c2aec150ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54a45f12 2021-05-28T00:04:05.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e134bb-e0a6-4e3b-95ad-c7c2aec150ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:05.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e134bb-e0a6-4e3b-95ad-c7c2aec150ba PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:08.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f96a3f6-81d8-425f-9ecb-cd4170a268d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:08.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f96a3f6-81d8-425f-9ecb-cd4170a268d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b1c321c 2021-05-28T00:04:08.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f96a3f6-81d8-425f-9ecb-cd4170a268d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:08.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f96a3f6-81d8-425f-9ecb-cd4170a268d5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:10.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:04:10.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:04:11.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d8b4a37-a18c-4ea4-a35d-259d52381fcc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d8b4a37-a18c-4ea4-a35d-259d52381fcc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c92a11f 2021-05-28T00:04:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d8b4a37-a18c-4ea4-a35d-259d52381fcc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d8b4a37-a18c-4ea4-a35d-259d52381fcc PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:12.291Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:04:12.292Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:04:12.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:04:12.306Z [cluster1-reconnection-1] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 128000 milliseconds 2021-05-28T00:04:12.677Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=febe5f95-0ec7-4dd1-aed4-e77c431abc20 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:12.676264Z InvocationID=7b22eada-a7db-4440-a004-1a14ba7900d0 LogTimestamp=2021-05-28T00:04:12.676830Z Entering 2021-05-28T00:04:12.677Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=febe5f95-0ec7-4dd1-aed4-e77c431abc20 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:12.676264Z InvocationID=7b22eada-a7db-4440-a004-1a14ba7900d0 LogTimestamp=2021-05-28T00:04:12.676830Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:04:12.776Z [qtp580718781-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:04:12.677Z InvocationID=16ab12dc-78d6-454d-903f-e6e081d18f4e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:04:12.868Z [qtp580718781-14] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:12.677Z InvocationID=16ab12dc-78d6-454d-903f-e6e081d18f4e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:04:12.868Z [qtp580718781-14] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:12.677Z InvocationID=16ab12dc-78d6-454d-903f-e6e081d18f4e ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:04:12.869Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:04:12.677Z InvocationID=16ab12dc-78d6-454d-903f-e6e081d18f4e ErrorCode=500 LogTimestamp=2021-05-28T00:04:12.869073Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:04:12.869Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:04:14.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b6b7a68-f6cb-4561-a81c-498f42d53c3c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:14.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b6b7a68-f6cb-4561-a81c-498f42d53c3c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@99aa430 2021-05-28T00:04:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b6b7a68-f6cb-4561-a81c-498f42d53c3c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b6b7a68-f6cb-4561-a81c-498f42d53c3c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:17.486Z [cluster2-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:04:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cc8c2d3-d015-482d-91f8-54d108bfe194 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cc8c2d3-d015-482d-91f8-54d108bfe194 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f3036de 2021-05-28T00:04:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cc8c2d3-d015-482d-91f8-54d108bfe194 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cc8c2d3-d015-482d-91f8-54d108bfe194 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4c9be5f-aaec-4ec5-9b5f-f0b5565e54b6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4c9be5f-aaec-4ec5-9b5f-f0b5565e54b6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54677f6 2021-05-28T00:04:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4c9be5f-aaec-4ec5-9b5f-f0b5565e54b6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4c9be5f-aaec-4ec5-9b5f-f0b5565e54b6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:22.981Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a2ae7cf8-f277-4733-88b2-ba06e3c3c68d InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:22.981279Z InvocationID=476352ea-6d75-43d2-a296-96285847424c LogTimestamp=2021-05-28T00:04:22.981350Z Entering 2021-05-28T00:04:22.982Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a2ae7cf8-f277-4733-88b2-ba06e3c3c68d InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:22.981279Z InvocationID=476352ea-6d75-43d2-a296-96285847424c LogTimestamp=2021-05-28T00:04:22.981350Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:04:23.095Z [qtp580718781-100] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:04:22.982Z InvocationID=91003024-8fbc-4a2d-bee4-ece6273a0b2d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:04:23.267Z [qtp580718781-100] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:22.982Z InvocationID=91003024-8fbc-4a2d-bee4-ece6273a0b2d ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:04:23.267Z [qtp580718781-100] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:22.982Z InvocationID=91003024-8fbc-4a2d-bee4-ece6273a0b2d ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:04:23.268Z [qtp580718781-100] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=286 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:04:22.982Z InvocationID=91003024-8fbc-4a2d-bee4-ece6273a0b2d ErrorCode=500 LogTimestamp=2021-05-28T00:04:23.268214Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:04:23.268Z [qtp580718781-100] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:04:23.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9407237a-4a49-45cf-bd3e-b40f8452a5f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:23.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9407237a-4a49-45cf-bd3e-b40f8452a5f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c45b418 2021-05-28T00:04:23.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9407237a-4a49-45cf-bd3e-b40f8452a5f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:23.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9407237a-4a49-45cf-bd3e-b40f8452a5f9 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:25.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:04:25.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:04:26.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce4481eb-c2ad-43ea-a27c-9a2ba3a90588 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:26.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce4481eb-c2ad-43ea-a27c-9a2ba3a90588 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40223884 2021-05-28T00:04:26.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce4481eb-c2ad-43ea-a27c-9a2ba3a90588 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:26.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce4481eb-c2ad-43ea-a27c-9a2ba3a90588 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:27.406Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:04:29.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f79ce65a-a6d6-4d84-adda-99c20571e36c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f79ce65a-a6d6-4d84-adda-99c20571e36c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3eda7098 2021-05-28T00:04:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f79ce65a-a6d6-4d84-adda-99c20571e36c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:29.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f79ce65a-a6d6-4d84-adda-99c20571e36c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3b83793-de59-4b20-9bf6-41a82141508e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3b83793-de59-4b20-9bf6-41a82141508e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8154972 2021-05-28T00:04:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3b83793-de59-4b20-9bf6-41a82141508e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:32.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3b83793-de59-4b20-9bf6-41a82141508e PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:33.476Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d9b5d307-81c1-4aec-9b76-dc3588ffc8f7 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:33.475935Z InvocationID=52157795-6b7a-4f07-8faf-27a72b06dab9 LogTimestamp=2021-05-28T00:04:33.476023Z Entering 2021-05-28T00:04:33.476Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d9b5d307-81c1-4aec-9b76-dc3588ffc8f7 InstanceID=4a79100b-1839-4dca-9d2a-0b197c591b51 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.173 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.173 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:04:33.475935Z InvocationID=52157795-6b7a-4f07-8faf-27a72b06dab9 LogTimestamp=2021-05-28T00:04:33.476023Z localAddr=10.242.102.221 remoteAddr=10.242.170.173 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:04:33.597Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:04:33.477Z InvocationID=54c13618-71a4-45a1-82fd-86f75c6caa32 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:04:33.670Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:33.477Z InvocationID=54c13618-71a4-45a1-82fd-86f75c6caa32 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:04:33.671Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:04:33.477Z InvocationID=54c13618-71a4-45a1-82fd-86f75c6caa32 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:04:33.672Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=195 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:04:33.477Z InvocationID=54c13618-71a4-45a1-82fd-86f75c6caa32 ErrorCode=500 LogTimestamp=2021-05-28T00:04:33.672003Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:04:33.672Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:04:35.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc3355b-d419-4f31-9851-8685c6c57dfb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc3355b-d419-4f31-9851-8685c6c57dfb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3941d91c 2021-05-28T00:04:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc3355b-d419-4f31-9851-8685c6c57dfb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:35.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc3355b-d419-4f31-9851-8685c6c57dfb PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:37.782Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:04:37.783Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to /10.242.170.166:9042, scheduling retry in 256000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:04:37.880Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:04:37.881Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 256000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:04:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6137080-e536-426d-be9c-7275bba1da7c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6137080-e536-426d-be9c-7275bba1da7c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68f630a8 2021-05-28T00:04:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6137080-e536-426d-be9c-7275bba1da7c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:38.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6137080-e536-426d-be9c-7275bba1da7c PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:40.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:04:40.518Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:04:41.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b892d4-e31d-4c68-9a3a-9bd9252699f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b892d4-e31d-4c68-9a3a-9bd9252699f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@545c39e 2021-05-28T00:04:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b892d4-e31d-4c68-9a3a-9bd9252699f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:41.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43b892d4-e31d-4c68-9a3a-9bd9252699f6 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:44.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5108301-771a-460c-ac48-b7ee910cfeb3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:44.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5108301-771a-460c-ac48-b7ee910cfeb3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27b9449d 2021-05-28T00:04:44.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5108301-771a-460c-ac48-b7ee910cfeb3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:44.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5108301-771a-460c-ac48-b7ee910cfeb3 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:47.492Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host /10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:04:47.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69fbed1-d2f5-4f84-ba97-db56d7cfd494 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:47.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69fbed1-d2f5-4f84-ba97-db56d7cfd494 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1087e04f 2021-05-28T00:04:47.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69fbed1-d2f5-4f84-ba97-db56d7cfd494 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:47.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69fbed1-d2f5-4f84-ba97-db56d7cfd494 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:50.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfcd7c88-0ad9-41ac-b829-f539067123ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:50.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfcd7c88-0ad9-41ac-b829-f539067123ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49ade187 2021-05-28T00:04:50.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfcd7c88-0ad9-41ac-b829-f539067123ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:50.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfcd7c88-0ad9-41ac-b829-f539067123ad PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:53.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=359f23c8-245f-450c-923c-02602a260660 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:53.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=359f23c8-245f-450c-923c-02602a260660 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62df821f 2021-05-28T00:04:53.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=359f23c8-245f-450c-923c-02602a260660 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:53.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=359f23c8-245f-450c-923c-02602a260660 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:55.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:04:55.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:04:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=421d3181-c17a-465c-bafa-9055b31da311 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=421d3181-c17a-465c-bafa-9055b31da311 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f2fbf75 2021-05-28T00:04:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=421d3181-c17a-465c-bafa-9055b31da311 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:56.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=421d3181-c17a-465c-bafa-9055b31da311 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:04:59.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4937339-a454-40f3-bee7-ed362692d037 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:04:59.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4937339-a454-40f3-bee7-ed362692d037 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2085a347 2021-05-28T00:04:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4937339-a454-40f3-bee7-ed362692d037 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:04:59.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4937339-a454-40f3-bee7-ed362692d037 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:02.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32e2d183-a117-40b1-a79d-6e0d368992f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:02.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32e2d183-a117-40b1-a79d-6e0d368992f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@602df09 2021-05-28T00:05:02.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32e2d183-a117-40b1-a79d-6e0d368992f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:02.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32e2d183-a117-40b1-a79d-6e0d368992f2 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:05.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0990bd5-3aa5-4879-bbfc-18b92a4d8769 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:05.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0990bd5-3aa5-4879-bbfc-18b92a4d8769 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18536aac 2021-05-28T00:05:05.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0990bd5-3aa5-4879-bbfc-18b92a4d8769 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:05.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0990bd5-3aa5-4879-bbfc-18b92a4d8769 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:08.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569faa0d-be97-4bb4-aae4-49018d6e6bcd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:08.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569faa0d-be97-4bb4-aae4-49018d6e6bcd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6af5bac4 2021-05-28T00:05:08.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569faa0d-be97-4bb4-aae4-49018d6e6bcd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:08.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569faa0d-be97-4bb4-aae4-49018d6e6bcd PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:10.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:05:10.517Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:05:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e84aa5ab-013f-4c81-9328-5bb63949efc3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e84aa5ab-013f-4c81-9328-5bb63949efc3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6767a346 2021-05-28T00:05:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e84aa5ab-013f-4c81-9328-5bb63949efc3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:11.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e84aa5ab-013f-4c81-9328-5bb63949efc3 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:12.291Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:05:12.292Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:05:12.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:05:14.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e7aa633-ae85-4e22-9d79-ef1c8c987696 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e7aa633-ae85-4e22-9d79-ef1c8c987696 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@778dd234 2021-05-28T00:05:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e7aa633-ae85-4e22-9d79-ef1c8c987696 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:14.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e7aa633-ae85-4e22-9d79-ef1c8c987696 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d080a616-a086-4b17-9dd5-e296e58284ee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d080a616-a086-4b17-9dd5-e296e58284ee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7488c063 2021-05-28T00:05:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d080a616-a086-4b17-9dd5-e296e58284ee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:17.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d080a616-a086-4b17-9dd5-e296e58284ee PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:20.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5681e338-9a4c-48d0-8397-dbf5b5cf10f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5681e338-9a4c-48d0-8397-dbf5b5cf10f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33ec67ac 2021-05-28T00:05:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5681e338-9a4c-48d0-8397-dbf5b5cf10f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:20.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5681e338-9a4c-48d0-8397-dbf5b5cf10f5 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:23.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8b2e7fc-442c-4d54-91d0-b9b2df272cdd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:23.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8b2e7fc-442c-4d54-91d0-b9b2df272cdd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@694b9f04 2021-05-28T00:05:23.729Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8b2e7fc-442c-4d54-91d0-b9b2df272cdd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:23.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8b2e7fc-442c-4d54-91d0-b9b2df272cdd PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:05:25.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:05:25.521Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:05:27.507Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:05:27.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19713a76-5686-47b0-bfeb-d5cfb98987fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:27.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19713a76-5686-47b0-bfeb-d5cfb98987fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70cefc6b 2021-05-28T00:05:27.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19713a76-5686-47b0-bfeb-d5cfb98987fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:27.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19713a76-5686-47b0-bfeb-d5cfb98987fe PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:05:30.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6204f58-3386-45f9-8ae9-44f94de4c5a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:30.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6204f58-3386-45f9-8ae9-44f94de4c5a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42f7b1c9 2021-05-28T00:05:30.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6204f58-3386-45f9-8ae9-44f94de4c5a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:30.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6204f58-3386-45f9-8ae9-44f94de4c5a1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:05:33.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f835192-f687-45a9-8957-29f71db75beb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:33.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f835192-f687-45a9-8957-29f71db75beb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12f66f3c 2021-05-28T00:05:33.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f835192-f687-45a9-8957-29f71db75beb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:33.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f835192-f687-45a9-8957-29f71db75beb PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:05:37.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f97af546-7627-41ae-8e55-bb918d3aafb1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:37.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f97af546-7627-41ae-8e55-bb918d3aafb1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@277071d9 2021-05-28T00:05:37.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f97af546-7627-41ae-8e55-bb918d3aafb1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:37.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f97af546-7627-41ae-8e55-bb918d3aafb1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:05:39.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d15551-fdac-4ce3-b265-638f4caa40e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:39.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d15551-fdac-4ce3-b265-638f4caa40e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20daecf8 2021-05-28T00:05:39.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d15551-fdac-4ce3-b265-638f4caa40e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:39.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45d15551-fdac-4ce3-b265-638f4caa40e2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:05:40.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:05:40.517Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:05:42.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61fc4cde-9d64-4a01-89d7-9b5ecc1e7447 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:05:42.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61fc4cde-9d64-4a01-89d7-9b5ecc1e7447 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f5c68d3 2021-05-28T00:05:42.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61fc4cde-9d64-4a01-89d7-9b5ecc1e7447 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:05:42.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61fc4cde-9d64-4a01-89d7-9b5ecc1e7447 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:06:08.497Z [main] WARN o.o.s.c.util.YamlToObjectConverter RequestID=null ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= The file /var/lib/jetty/config/catalog-be/neo4j-errors-configuration.yaml cannot be found. Ignore reading configuration. 2021-05-28T00:06:08.660Z [main] INFO o.o.s.be.listen.BEAppContextListener RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= ASDC version from MANIFEST is 1.8.5 2021-05-28T00:06:08.708Z [main] INFO o.o.s.b.m.BeMonitoringService RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= BE monitoring service is disabled 2021-05-28T00:06:08.708Z [main] INFO o.s.web.context.ContextLoader RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Root WebApplicationContext: initialization started 2021-05-28T00:06:10.784Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= ** JanusGraphClient created 2021-05-28T00:06:10.788Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= ** createGraph started ** 2021-05-28T00:06:10.788Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= ** open graph with /var/lib/jetty/config/catalog-be/janusgraph.properties started 2021-05-28T00:06:10.788Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= openGraph : try to load file /var/lib/jetty/config/catalog-be/janusgraph.properties 2021-05-28T00:06:11.177Z [main] INFO com.datastax.driver.core RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= DataStax Java driver 3.8.0 for Apache Cassandra 2021-05-28T00:06:11.183Z [main] INFO c.d.driver.core.GuavaCompatibility RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Detected Guava >= 19 in the classpath, using modern compatibility layer 2021-05-28T00:06:11.844Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Using native clock to generate timestamps. 2021-05-28T00:06:12.057Z [main] INFO com.datastax.driver.core.NettyUtil RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Found Netty's native epoll transport in the classpath, using it 2021-05-28T00:06:13.842Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-05-28T00:06:13.844Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host cassandra.onap/10.242.170.166:9042 added 2021-05-28T00:06:13.844Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host /10.242.140.16:9042 added 2021-05-28T00:06:13.844Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host /10.242.76.215:9042 added 2021-05-28T00:06:15.498Z [main] WARN o.j.g.c.GraphDatabaseConfiguration RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Local setting storage.cql.replication-strategy-options=[Pod, 3] (Type: FIXED) is overridden by globally managed value ([Pod, 3]). Use the ManagementSystem interface instead of the local configuration to control this setting. 2021-05-28T00:06:17.657Z [main] INFO o.j.g.c.GraphDatabaseConfiguration RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Generated unique-instance-id=0af266dd253-dev-sdc-be-7697c4f957-9fbvk1 2021-05-28T00:06:17.679Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Using native clock to generate timestamps. 2021-05-28T00:06:18.280Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-05-28T00:06:18.280Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host cassandra.onap/10.242.170.166:9042 added 2021-05-28T00:06:18.281Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host /10.242.140.16:9042 added 2021-05-28T00:06:18.281Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host /10.242.76.215:9042 added 2021-05-28T00:06:19.186Z [main] INFO org.janusgraph.diskstorage.Backend RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Initiated backend operations thread pool of size 2 2021-05-28T00:06:21.680Z [main] INFO o.j.diskstorage.log.kcvs.KCVSLog RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Loaded unidentified ReadMarker start time 2021-05-28T00:06:21.680279Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@3fe1b278 2021-05-28T00:06:22.074Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= ** JanusGraph graph created 2021-05-28T00:06:22.503Z [main] INFO o.o.s.b.d.j.JanusGraphGenericDao RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= ** JanusGraphGenericDao created 2021-05-28T00:06:22.719Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:06:22.809Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= All sdc keyspaces are : [sdcartifact, sdcrepository, sdcaudit] 2021-05-28T00:06:22.810Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Connecting to node: [cassandra.onap] port: 9042. 2021-05-28T00:06:22.811Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Connection timeout in seconds : 120 2021-05-28T00:06:22.811Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2021-05-28T00:06:22.811Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2021-05-28T00:06:22.811Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2021-05-28T00:06:22.812Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Using native clock to generate timestamps. 2021-05-28T00:06:22.812Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= ** cluster created 2021-05-28T00:06:22.812Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= creating cluster for Cassandra Health Check. 2021-05-28T00:06:23.701Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-05-28T00:06:23.701Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host cassandra.onap/10.242.140.16:9042 added 2021-05-28T00:06:23.701Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host cassandra.onap/10.242.170.166:9042 added 2021-05-28T00:06:23.701Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= New Cassandra host cassandra.onap/10.242.76.215:9042 added 2021-05-28T00:06:23.701Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Cluster Metadata: com.datastax.driver.core.Metadata@608f79a8 2021-05-28T00:06:23.701Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= keyspace : sdcartifact , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2021-05-28T00:06:23.702Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= keyspace : sdcrepository , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2021-05-28T00:06:23.702Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= keyspace : sdcaudit , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2021-05-28T00:06:23.702Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= maxReplication Factor is: 3 2021-05-28T00:06:23.702Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= localQuorum is: 2 2021-05-28T00:06:23.702Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=WARN ServerIPAddress=10.242.102.221 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity= Health Check formula : Replication Factor – Local_Quorum = 1 2021-05-28T00:06:23.718Z [main] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null ECOMP Portal health check configuration is missing. 2021-05-28T00:06:23.731Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Enter init method of SwitchoverDetector 2021-05-28T00:06:23.731Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null switchover detector service is disabled 2021-05-28T00:06:24.557Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null creating cluster for Cassandra for monitoring. 2021-05-28T00:06:24.557Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null The cassandra session is com.datastax.driver.core.SessionManager@67726dd1 2021-05-28T00:06:24.557Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null The number of cassandra nodes is:3 2021-05-28T00:06:24.558Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null The cassandra down nodes number is 0 2021-05-28T00:06:25.196Z [main] INFO o.o.s.be.dao.jsongraph.JanusGraphDao RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null ** JanusGraphDao created 2021-05-28T00:06:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8a1829-a9d9-4309-86ed-cfc1a0519779 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:25.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8a1829-a9d9-4309-86ed-cfc1a0519779 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d52ae42 2021-05-28T00:06:25.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8a1829-a9d9-4309-86ed-cfc1a0519779 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:25.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8a1829-a9d9-4309-86ed-cfc1a0519779 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:27.416Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2021-05-28T00:06:27.416Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2021-05-28T00:06:27.416Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2021-05-28T00:06:27.416Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Using native clock to generate timestamps. 2021-05-28T00:06:27.417Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null ** CassandraClient created 2021-05-28T00:06:27.626Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-05-28T00:06:27.626Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null New Cassandra host cassandra.onap/10.242.140.16:9042 added 2021-05-28T00:06:27.627Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null New Cassandra host cassandra.onap/10.242.170.166:9042 added 2021-05-28T00:06:27.627Z [main] INFO com.datastax.driver.core.Cluster RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null New Cassandra host cassandra.onap/10.242.76.215:9042 added 2021-05-28T00:06:28.641Z [main] INFO o.o.s.b.d.c.AuditCassandraDao RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null ** AuditCassandraDao created 2021-05-28T00:06:28.810Z [main] INFO o.o.s.b.c.d.e.DmaapClientFactory RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1622160351, consumerId=sdc-AUTO1-1622160351, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/var/lib/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-05-28T00:06:28.873Z [main] ERROR o.o.s.be.catalog.impl.DmaapProducer RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-05-28T00:06:28.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154b5003-ad3f-4522-9c4c-06878054bd47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:28.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154b5003-ad3f-4522-9c4c-06878054bd47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73205e41 2021-05-28T00:06:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154b5003-ad3f-4522-9c4c-06878054bd47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154b5003-ad3f-4522-9c4c-06878054bd47 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:30.678Z [main] INFO o.o.s.b.d.c.ArtifactCassandraDao RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null ** ArtifactCassandraDao created 2021-05-28T00:06:32.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d502655a-bbdf-4e33-967b-9a433a3e6a2f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:32.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d502655a-bbdf-4e33-967b-9a433a3e6a2f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22728642 2021-05-28T00:06:32.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d502655a-bbdf-4e33-967b-9a433a3e6a2f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:32.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d502655a-bbdf-4e33-967b-9a433a3e6a2f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:32.361Z [main] INFO o.o.s.b.d.c.SdcSchemaFilesCassandraDao RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null ** SdcSchemaFilesCassandraDao created 2021-05-28T00:06:32.622Z [main] WARN o.o.s.b.c.s.CertificateManagerImpl RequestID=25d80915-55ad-4727-8db0-70570057c300 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Environment variable 'SDC_CERT_DIR' was not provided. Could not load certificates. 2021-05-28T00:06:34.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3e1f272-183f-497d-b9fe-2a794ba5bd76 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:34.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3e1f272-183f-497d-b9fe-2a794ba5bd76 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2287a4ec 2021-05-28T00:06:34.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3e1f272-183f-497d-b9fe-2a794ba5bd76 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:34.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3e1f272-183f-497d-b9fe-2a794ba5bd76 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:35.476Z [main] INFO o.o.s.b.c.d.e.EnvironmentsEngine RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE Environments engine is disabled 2021-05-28T00:06:35.532Z [pool-14-thread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/topics (anonymous) ... 2021-05-28T00:06:35.546Z [pool-14-thread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:06:35.864Z [pool-14-thread-1] INFO com.datastax.driver.core.utils.UUIDs PID obtained through native call to getpid(): 253 2021-05-28T00:06:36.065Z [pool-14-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "ALREADY_EXISTS" PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/producers/iPIxkpAMI8qTcQj8 will send credentials over a clear channel. 2021-05-28T00:06:36.065Z [pool-14-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "ALREADY_EXISTS" PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/producers/iPIxkpAMI8qTcQj8 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:06:36.973Z [pool-14-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "ALREADY_EXISTS" \t--> HTTP/1.1 200 OK 2021-05-28T00:06:37.061Z [pool-14-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "ALREADY_EXISTS" PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/iPIxkpAMI8qTcQj8 will send credentials over a clear channel. 2021-05-28T00:06:37.061Z [pool-14-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "ALREADY_EXISTS" PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/iPIxkpAMI8qTcQj8 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:06:37.138Z [pool-14-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "ALREADY_EXISTS" \t--> HTTP/1.1 200 OK 2021-05-28T00:06:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:06:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:06:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:06:37.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc94e284-e5f6-4ae9-aae8-0dbb86f27962 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:37.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc94e284-e5f6-4ae9-aae8-0dbb86f27962 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ea3d53b 2021-05-28T00:06:37.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc94e284-e5f6-4ae9-aae8-0dbb86f27962 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:37.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc94e284-e5f6-4ae9-aae8-0dbb86f27962 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:39.721Z [main] INFO o.o.s.b.c.s.AsdcComponentsCleanerTask RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Enter init method of AsdcComponentsCleaner 2021-05-28T00:06:39.722Z [main] INFO o.o.s.b.c.s.AsdcComponentsCleanerTask RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk End init method of AsdcComponentsCleaner 2021-05-28T00:06:39.785Z [main] INFO o.s.b.f.a.AutowiredAnnotationBeanPostProcessor RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Autowired annotation is not supported on static methods: public static void org.openecomp.sdc.be.components.impl.ImportUtils.setComponentsUtils(org.openecomp.sdc.be.impl.ComponentsUtils) 2021-05-28T00:06:40.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7245a880-fc99-4b30-b91b-9e57c29d8c63 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:40.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7245a880-fc99-4b30-b91b-9e57c29d8c63 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f004ef8 2021-05-28T00:06:40.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7245a880-fc99-4b30-b91b-9e57c29d8c63 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:40.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7245a880-fc99-4b30-b91b-9e57c29d8c63 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:41.065Z [main] INFO o.o.s.b.d.cassandra.FeatureToggleDao RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ** FeatureToggleDao created 2021-05-28T00:06:41.547Z [main] INFO o.s.web.context.ContextLoader RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Root WebApplicationContext initialized in 32839 ms 2021-05-28T00:06:44.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e88af5-e195-45fa-8b40-65a2f641307e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:44.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e88af5-e195-45fa-8b40-65a2f641307e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@397689cc 2021-05-28T00:06:44.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e88af5-e195-45fa-8b40-65a2f641307e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:44.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e88af5-e195-45fa-8b40-65a2f641307e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:44.938Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2021-05-28T00:06:44.958Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2021-05-28T00:06:44.990Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2021-05-28T00:06:47.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db2fb9ef-ab3a-44e3-bd97-6ef3b4d8309c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:47.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db2fb9ef-ab3a-44e3-bd97-6ef3b4d8309c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26c9faca 2021-05-28T00:06:47.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db2fb9ef-ab3a-44e3-bd97-6ef3b4d8309c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:47.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db2fb9ef-ab3a-44e3-bd97-6ef3b4d8309c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:48.281Z [main] INFO o.h.validator.internal.util.Version RequestID=9b60d69e-5085-4631-b1df-b1f0e9871764 PartnerName=SDC.BE ServerIPAddress=10.242.102.221 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk HV000001: Hibernate Validator 5.3.6.Final 2021-05-28T00:06:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1821c9-adab-40a1-a4cf-cef15d9dcc8e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1821c9-adab-40a1-a4cf-cef15d9dcc8e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4623e925 2021-05-28T00:06:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1821c9-adab-40a1-a4cf-cef15d9dcc8e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1821c9-adab-40a1-a4cf-cef15d9dcc8e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:50.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:06:50.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:06:50.515Z [UEB-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL ErrorCategory=INFO RequestID=null ServiceName=uebHealthCheck TargetServiceName=null ErrorCode=200 TargetEntity=uebHealthCheck ASDC Backend connection recovery to UEB Cluster. 2021-05-28T00:06:52.051Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:06:52.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6024c8c-2ae7-4fdf-874c-dc43d0fe4262 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:52.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6024c8c-2ae7-4fdf-874c-dc43d0fe4262 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4bd91897 2021-05-28T00:06:52.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6024c8c-2ae7-4fdf-874c-dc43d0fe4262 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:52.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6024c8c-2ae7-4fdf-874c-dc43d0fe4262 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:52.861Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b6024c8c-2ae7-4fdf-874c-dc43d0fe4262 ErrorCategory=INFO ServiceName=BE Component DE state changed from UNKNOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component DE state changed from UNKNOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:06:52.862Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b6024c8c-2ae7-4fdf-874c-dc43d0fe4262 ErrorCategory=INFO ServiceName=BE Component DE state changed from UNKNOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:06:53.058Z [qtp580718781-97] INFO com.jcabi.aspects.aj.NamedThreads RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/healthCheck EntryTimestamp=2021-05-28T00:06:52.957Z InvocationID=207441f8-b059-4c99-8950-37e289b4221f PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk jcabi-aspects 0.20.1/f0aff93 started new daemon thread jcabi-loggable for watching of @Loggable annotated methods 2021-05-28T00:06:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24080f0-4548-4832-b6e9-f1260b07e6e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24080f0-4548-4832-b6e9-f1260b07e6e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46ecd0c9 2021-05-28T00:06:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24080f0-4548-4832-b6e9-f1260b07e6e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24080f0-4548-4832-b6e9-f1260b07e6e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:06:58.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffa1f251-ee5e-4cd4-a9e4-b9fa858f5064 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:06:58.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffa1f251-ee5e-4cd4-a9e4-b9fa858f5064 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f1a7d74 2021-05-28T00:06:58.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffa1f251-ee5e-4cd4-a9e4-b9fa858f5064 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:06:58.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffa1f251-ee5e-4cd4-a9e4-b9fa858f5064 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:01.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd78714f-5bd8-47d2-873b-c69c8677430d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:01.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd78714f-5bd8-47d2-873b-c69c8677430d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7629d373 2021-05-28T00:07:01.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd78714f-5bd8-47d2-873b-c69c8677430d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:01.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd78714f-5bd8-47d2-873b-c69c8677430d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:05.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac225d4b-8a77-409a-8aed-f9479c4259ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:05.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac225d4b-8a77-409a-8aed-f9479c4259ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75376081 2021-05-28T00:07:05.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac225d4b-8a77-409a-8aed-f9479c4259ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:05.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac225d4b-8a77-409a-8aed-f9479c4259ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:07:05.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:07:07.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=047fe3fb-7bac-4858-b01c-0d90921b41f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:07.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=047fe3fb-7bac-4858-b01c-0d90921b41f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55fd9c76 2021-05-28T00:07:07.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=047fe3fb-7bac-4858-b01c-0d90921b41f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:07.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=047fe3fb-7bac-4858-b01c-0d90921b41f3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a247460c-23a6-48fd-9ad4-71f5b8a7cb3a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a247460c-23a6-48fd-9ad4-71f5b8a7cb3a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29f0aa60 2021-05-28T00:07:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a247460c-23a6-48fd-9ad4-71f5b8a7cb3a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a247460c-23a6-48fd-9ad4-71f5b8a7cb3a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:14.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ca2e19-01fb-4bbc-9a5e-c8ffdc762de4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:14.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ca2e19-01fb-4bbc-9a5e-c8ffdc762de4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@117a5666 2021-05-28T00:07:14.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ca2e19-01fb-4bbc-9a5e-c8ffdc762de4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:14.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ca2e19-01fb-4bbc-9a5e-c8ffdc762de4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:17.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0628b996-c923-462a-956c-afe90cce7d6c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:17.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0628b996-c923-462a-956c-afe90cce7d6c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f870fa0 2021-05-28T00:07:17.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0628b996-c923-462a-956c-afe90cce7d6c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:17.082Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0628b996-c923-462a-956c-afe90cce7d6c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:19.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=573e9194-6532-400d-96db-0c01f609ee4b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:19.983Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=573e9194-6532-400d-96db-0c01f609ee4b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66a0260a 2021-05-28T00:07:19.983Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=573e9194-6532-400d-96db-0c01f609ee4b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:19.983Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=573e9194-6532-400d-96db-0c01f609ee4b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:20.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:07:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:07:23.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7502ad3b-ab27-4693-a0aa-d6b17e87f26e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:23.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7502ad3b-ab27-4693-a0aa-d6b17e87f26e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4aaf3314 2021-05-28T00:07:23.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7502ad3b-ab27-4693-a0aa-d6b17e87f26e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:23.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7502ad3b-ab27-4693-a0aa-d6b17e87f26e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:26.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=662576f7-ed04-4a65-abfd-0e99bf27413a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:26.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=662576f7-ed04-4a65-abfd-0e99bf27413a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9a59541 2021-05-28T00:07:26.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=662576f7-ed04-4a65-abfd-0e99bf27413a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:26.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=662576f7-ed04-4a65-abfd-0e99bf27413a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:28.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f9339e-0e97-40dc-8d27-c6f78c9b063a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:28.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f9339e-0e97-40dc-8d27-c6f78c9b063a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6eb871da 2021-05-28T00:07:28.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f9339e-0e97-40dc-8d27-c6f78c9b063a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:28.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f9339e-0e97-40dc-8d27-c6f78c9b063a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:32.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31373672-8ac9-4c58-b277-38bec4abd7d2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:32.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31373672-8ac9-4c58-b277-38bec4abd7d2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9cf9a71 2021-05-28T00:07:32.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31373672-8ac9-4c58-b277-38bec4abd7d2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:32.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31373672-8ac9-4c58-b277-38bec4abd7d2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:35.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d7502b-2b28-4887-97ee-121b35bcab4e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:35.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d7502b-2b28-4887-97ee-121b35bcab4e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3acb2fcf 2021-05-28T00:07:35.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d7502b-2b28-4887-97ee-121b35bcab4e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:35.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d7502b-2b28-4887-97ee-121b35bcab4e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:07:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:07:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:07:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:07:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:07:39.196Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e25f3bad-d78d-4ec6-87dc-ff5c87b235d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:39.196Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e25f3bad-d78d-4ec6-87dc-ff5c87b235d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f61aa4a 2021-05-28T00:07:39.196Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e25f3bad-d78d-4ec6-87dc-ff5c87b235d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:39.197Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e25f3bad-d78d-4ec6-87dc-ff5c87b235d3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:41.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69eb9f89-a207-4bea-a993-f5d0dba2e4e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:41.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69eb9f89-a207-4bea-a993-f5d0dba2e4e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32fa3edd 2021-05-28T00:07:41.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69eb9f89-a207-4bea-a993-f5d0dba2e4e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:41.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69eb9f89-a207-4bea-a993-f5d0dba2e4e4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:45.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b500d7c4-ae8c-45e6-b08d-47ec85957cef PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:45.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b500d7c4-ae8c-45e6-b08d-47ec85957cef PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@471326e1 2021-05-28T00:07:45.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b500d7c4-ae8c-45e6-b08d-47ec85957cef PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:45.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b500d7c4-ae8c-45e6-b08d-47ec85957cef PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:47.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c678c2b9-4884-4ef4-b6db-8adfa84af32f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:47.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c678c2b9-4884-4ef4-b6db-8adfa84af32f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2445cc9b 2021-05-28T00:07:47.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c678c2b9-4884-4ef4-b6db-8adfa84af32f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:47.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c678c2b9-4884-4ef4-b6db-8adfa84af32f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:50.019Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f3b5c1b-b752-4561-a37e-7968c94c7cdc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:50.019Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f3b5c1b-b752-4561-a37e-7968c94c7cdc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@366411a3 2021-05-28T00:07:50.019Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f3b5c1b-b752-4561-a37e-7968c94c7cdc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:50.019Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f3b5c1b-b752-4561-a37e-7968c94c7cdc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:07:50.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:07:52.265Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:07:52.913Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8d8dcd0-05fa-4808-852e-876abab65d01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:52.913Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8d8dcd0-05fa-4808-852e-876abab65d01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@550bdd64 2021-05-28T00:07:52.914Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8d8dcd0-05fa-4808-852e-876abab65d01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:52.914Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8d8dcd0-05fa-4808-852e-876abab65d01 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:57.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fd1829b-da22-4551-b659-ed5f94a37585 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:57.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fd1829b-da22-4551-b659-ed5f94a37585 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bf1cd0a 2021-05-28T00:07:57.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fd1829b-da22-4551-b659-ed5f94a37585 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:57.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fd1829b-da22-4551-b659-ed5f94a37585 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:07:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8ff11a-562b-4615-bd26-6da69c6151f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:07:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8ff11a-562b-4615-bd26-6da69c6151f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15af9fb 2021-05-28T00:07:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8ff11a-562b-4615-bd26-6da69c6151f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:07:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8ff11a-562b-4615-bd26-6da69c6151f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:01.992Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33cd247d-1132-4e7c-9809-83dbb1612362 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:01.992Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33cd247d-1132-4e7c-9809-83dbb1612362 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1725097a 2021-05-28T00:08:01.992Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33cd247d-1132-4e7c-9809-83dbb1612362 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:01.992Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33cd247d-1132-4e7c-9809-83dbb1612362 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:08:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:08:05.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b126179-04a6-4103-988c-f2d6f54d731a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:05.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b126179-04a6-4103-988c-f2d6f54d731a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c4e1607 2021-05-28T00:08:05.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b126179-04a6-4103-988c-f2d6f54d731a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:05.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b126179-04a6-4103-988c-f2d6f54d731a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:08.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd3e5f4-5af5-4df3-a465-55a61e5d3fa1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:08.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd3e5f4-5af5-4df3-a465-55a61e5d3fa1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19520119 2021-05-28T00:08:08.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd3e5f4-5af5-4df3-a465-55a61e5d3fa1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:08.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd3e5f4-5af5-4df3-a465-55a61e5d3fa1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:11.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3772574-f53c-4d88-8c32-be4ee9cca437 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:11.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3772574-f53c-4d88-8c32-be4ee9cca437 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f91b6e9 2021-05-28T00:08:11.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3772574-f53c-4d88-8c32-be4ee9cca437 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:11.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3772574-f53c-4d88-8c32-be4ee9cca437 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:13.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31cd59ea-0781-4bbf-aab8-2e018ad4aeee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:13.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31cd59ea-0781-4bbf-aab8-2e018ad4aeee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70c008b8 2021-05-28T00:08:13.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31cd59ea-0781-4bbf-aab8-2e018ad4aeee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:13.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31cd59ea-0781-4bbf-aab8-2e018ad4aeee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:17.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:17.472Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:08:18.502Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e43ecd59-ae4d-440b-9303-a5ee5e5e7a6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:18.502Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e43ecd59-ae4d-440b-9303-a5ee5e5e7a6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cd2d689 2021-05-28T00:08:18.502Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e43ecd59-ae4d-440b-9303-a5ee5e5e7a6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:18.502Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e43ecd59-ae4d-440b-9303-a5ee5e5e7a6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:19.770Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:08:20.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782bc9af-4e0d-4ba9-af00-96dd5c75f595 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:20.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782bc9af-4e0d-4ba9-af00-96dd5c75f595 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@150156c3 2021-05-28T00:08:20.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782bc9af-4e0d-4ba9-af00-96dd5c75f595 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:20.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782bc9af-4e0d-4ba9-af00-96dd5c75f595 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:20.167Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=782bc9af-4e0d-4ba9-af00-96dd5c75f595 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:08:20.167Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=782bc9af-4e0d-4ba9-af00-96dd5c75f595 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:08:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:08:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:08:23.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37600d89-92a0-4417-9332-b39d41bfcb6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:23.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37600d89-92a0-4417-9332-b39d41bfcb6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57e58c74 2021-05-28T00:08:23.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37600d89-92a0-4417-9332-b39d41bfcb6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:23.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37600d89-92a0-4417-9332-b39d41bfcb6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:23.991Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=37600d89-92a0-4417-9332-b39d41bfcb6b ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:08:23.992Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=37600d89-92a0-4417-9332-b39d41bfcb6b ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:08:26.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8730b8f-fc32-4db3-baf9-3f4325a52a27 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:26.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8730b8f-fc32-4db3-baf9-3f4325a52a27 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2710dbf8 2021-05-28T00:08:26.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8730b8f-fc32-4db3-baf9-3f4325a52a27 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:26.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8730b8f-fc32-4db3-baf9-3f4325a52a27 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:29.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:29.471Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:08:31.690Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:08:31.994Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8834ec7-d75c-4ecf-984a-2c260d2ebed6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:31.995Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8834ec7-d75c-4ecf-984a-2c260d2ebed6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d7c370d 2021-05-28T00:08:31.995Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8834ec7-d75c-4ecf-984a-2c260d2ebed6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:31.995Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8834ec7-d75c-4ecf-984a-2c260d2ebed6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:32.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438b5363-3572-4120-8c68-965316279cbc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:32.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438b5363-3572-4120-8c68-965316279cbc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32d4db2f 2021-05-28T00:08:32.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438b5363-3572-4120-8c68-965316279cbc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:32.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=438b5363-3572-4120-8c68-965316279cbc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:35.286Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1678485-81de-43a1-b5ab-bc885fe203a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:35.286Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1678485-81de-43a1-b5ab-bc885fe203a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49a9672d 2021-05-28T00:08:35.286Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1678485-81de-43a1-b5ab-bc885fe203a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:35.286Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1678485-81de-43a1-b5ab-bc885fe203a3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:08:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:08:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:08:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:08:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:08:40.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15f767c8-f86f-44c6-a6cc-c9d15ca2f829 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:40.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15f767c8-f86f-44c6-a6cc-c9d15ca2f829 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6847bafb 2021-05-28T00:08:40.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15f767c8-f86f-44c6-a6cc-c9d15ca2f829 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:40.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15f767c8-f86f-44c6-a6cc-c9d15ca2f829 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:41.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:08:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:08:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:08:52.118Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:08:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:08:59.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6910e028-92d7-4b1c-8f77-89cbf20bb2b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:08:59.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6910e028-92d7-4b1c-8f77-89cbf20bb2b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24cf2ca1 2021-05-28T00:08:59.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6910e028-92d7-4b1c-8f77-89cbf20bb2b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:08:59.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6910e028-92d7-4b1c-8f77-89cbf20bb2b8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:08:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:09:02.674Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:09:04.375Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:09:04.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19ced5ab-95d3-4288-a3dc-7542e8a62d1d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:04.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19ced5ab-95d3-4288-a3dc-7542e8a62d1d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32441f8c 2021-05-28T00:09:04.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19ced5ab-95d3-4288-a3dc-7542e8a62d1d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:04.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19ced5ab-95d3-4288-a3dc-7542e8a62d1d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:09:04.382Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=19ced5ab-95d3-4288-a3dc-7542e8a62d1d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:09:04.382Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=19ced5ab-95d3-4288-a3dc-7542e8a62d1d ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:09:05.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85715706-13e5-464d-a3fe-3507080a213e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:05.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85715706-13e5-464d-a3fe-3507080a213e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6781eb5d 2021-05-28T00:09:05.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85715706-13e5-464d-a3fe-3507080a213e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:05.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85715706-13e5-464d-a3fe-3507080a213e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:09:05.086Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=85715706-13e5-464d-a3fe-3507080a213e ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:09:05.086Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=85715706-13e5-464d-a3fe-3507080a213e ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:09:05.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:09:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:09:05.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd93824-9133-435c-b998-7cb44e0f50b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:05.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd93824-9133-435c-b998-7cb44e0f50b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a766928 2021-05-28T00:09:05.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd93824-9133-435c-b998-7cb44e0f50b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:05.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edd93824-9133-435c-b998-7cb44e0f50b0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:09:05.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cba750-36e8-4d79-aff2-76e9f44c2286 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:05.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cba750-36e8-4d79-aff2-76e9f44c2286 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e11abf4 2021-05-28T00:09:05.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cba750-36e8-4d79-aff2-76e9f44c2286 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:05.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cba750-36e8-4d79-aff2-76e9f44c2286 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:09:06.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61c3ee08-c69b-4ae2-80ed-36b15795dde9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:06.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61c3ee08-c69b-4ae2-80ed-36b15795dde9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cbb7bc5 2021-05-28T00:09:06.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61c3ee08-c69b-4ae2-80ed-36b15795dde9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:06.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61c3ee08-c69b-4ae2-80ed-36b15795dde9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:07.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e621cfbb-f882-4f2b-b9cd-45910e6f5b25 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:07.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e621cfbb-f882-4f2b-b9cd-45910e6f5b25 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4408b60c 2021-05-28T00:09:07.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e621cfbb-f882-4f2b-b9cd-45910e6f5b25 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:07.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e621cfbb-f882-4f2b-b9cd-45910e6f5b25 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:08.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130271f-e052-43f4-8f62-edb1b4ddd325 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:08.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130271f-e052-43f4-8f62-edb1b4ddd325 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4dbeb88c 2021-05-28T00:09:08.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130271f-e052-43f4-8f62-edb1b4ddd325 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:08.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130271f-e052-43f4-8f62-edb1b4ddd325 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:08.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aca357a-6587-4785-aa3f-1d2a9e62a299 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:08.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aca357a-6587-4785-aa3f-1d2a9e62a299 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ec3f4c4 2021-05-28T00:09:08.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aca357a-6587-4785-aa3f-1d2a9e62a299 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:08.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aca357a-6587-4785-aa3f-1d2a9e62a299 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:09.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcea66-eda0-4f65-8666-9c65c6c67d4e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:09.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcea66-eda0-4f65-8666-9c65c6c67d4e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9d1cd8a 2021-05-28T00:09:09.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcea66-eda0-4f65-8666-9c65c6c67d4e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:09.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcea66-eda0-4f65-8666-9c65c6c67d4e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b910d75-019f-48fb-ae8a-a804e76e84d2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b910d75-019f-48fb-ae8a-a804e76e84d2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8e8a26a 2021-05-28T00:09:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b910d75-019f-48fb-ae8a-a804e76e84d2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b910d75-019f-48fb-ae8a-a804e76e84d2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:13.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aab5e310-49b9-47fd-b8a5-4fc5b1ec9c6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:13.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aab5e310-49b9-47fd-b8a5-4fc5b1ec9c6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@302a1123 2021-05-28T00:09:13.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aab5e310-49b9-47fd-b8a5-4fc5b1ec9c6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:13.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aab5e310-49b9-47fd-b8a5-4fc5b1ec9c6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:17.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08065701-d309-437f-ae86-7a7076d2a96c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:17.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08065701-d309-437f-ae86-7a7076d2a96c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@512ea636 2021-05-28T00:09:17.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08065701-d309-437f-ae86-7a7076d2a96c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:17.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08065701-d309-437f-ae86-7a7076d2a96c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:19.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8733d3f6-529f-4aff-aa35-91ca04881ec6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:19.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8733d3f6-529f-4aff-aa35-91ca04881ec6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56148c04 2021-05-28T00:09:19.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8733d3f6-529f-4aff-aa35-91ca04881ec6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:19.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8733d3f6-529f-4aff-aa35-91ca04881ec6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:09:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:09:21.688Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b7f0ad7c-908d-4f2a-8d46-f7fdf298a6f3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=1 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:21.682177Z InvocationID=a9cc91a1-f00b-4b77-b081-79c2a9197c9f LogTimestamp=2021-05-28T00:09:21.683858Z Entering 2021-05-28T00:09:21.689Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b7f0ad7c-908d-4f2a-8d46-f7fdf298a6f3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=1 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:21.682177Z InvocationID=a9cc91a1-f00b-4b77-b081-79c2a9197c9f LogTimestamp=2021-05-28T00:09:21.683858Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:09:21.878Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=189 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:09:21.689Z InvocationID=258ca43f-ee24-40df-8732-6a1e2876aa4b LogTimestamp=2021-05-28T00:09:21.878211Z ResponseDesc=OK Exiting. 2021-05-28T00:09:21.880Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:09:22.073Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:22.073563Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:22.073Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:22.073563Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:22.075Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:09:22.264Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3b99b540-3af2-47c9-a1a3-d73ff5a7cbf0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:22.263895Z InvocationID=17044bdc-c6a7-486c-a43c-2644a1dd93a6 LogTimestamp=2021-05-28T00:09:22.263994Z Entering 2021-05-28T00:09:22.264Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3b99b540-3af2-47c9-a1a3-d73ff5a7cbf0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:22.263895Z InvocationID=17044bdc-c6a7-486c-a43c-2644a1dd93a6 LogTimestamp=2021-05-28T00:09:22.263994Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:09:22.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=056f1179-ff84-457d-a4f9-866b36687db9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:22.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=056f1179-ff84-457d-a4f9-866b36687db9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b229f5e 2021-05-28T00:09:22.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=056f1179-ff84-457d-a4f9-866b36687db9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:22.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=056f1179-ff84-457d-a4f9-866b36687db9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:22.883Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:09:22.264Z InvocationID=58ce30dd-8c32-4d6b-97d1-3b01d704fc59 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@7721eef2', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 375ms 2021-05-28T00:09:22.892Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=627 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:09:22.264Z InvocationID=58ce30dd-8c32-4d6b-97d1-3b01d704fc59 ErrorCode=409 LogTimestamp=2021-05-28T00:09:22.891539Z ResponseDesc=Conflict Exiting. 2021-05-28T00:09:22.893Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:09:22.976Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:22.976390Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:22.976Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:22.976390Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:22.977Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:09:23.275Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e793f2ae-25b3-4a30-8801-11092ded5524 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:23.275068Z InvocationID=cd74ae13-41b5-4620-97ae-86f695517c96 LogTimestamp=2021-05-28T00:09:23.275228Z Entering 2021-05-28T00:09:23.275Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e793f2ae-25b3-4a30-8801-11092ded5524 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:23.275068Z InvocationID=cd74ae13-41b5-4620-97ae-86f695517c96 LogTimestamp=2021-05-28T00:09:23.275228Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:09:23.559Z [qtp580718781-16] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:09:23.276Z InvocationID=222dd6d9-b881-4fbd-9a5a-336ea273464c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@7721eef2', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 281ms 2021-05-28T00:09:23.561Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=284 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:09:23.276Z InvocationID=222dd6d9-b881-4fbd-9a5a-336ea273464c ErrorCode=400 LogTimestamp=2021-05-28T00:09:23.560941Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:09:23.561Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:09:23.580Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=8746 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:09:14.834Z InvocationID=d36f1255-6ba3-4bfe-aaf0-9e4ffa0c3349 ErrorCode=500 LogTimestamp=2021-05-28T00:09:23.580114Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:23.581Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:09:23.677Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=95015b86-17f9-4111-ad24-ebc1677ac083 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:23.676936Z InvocationID=60b96a36-9eb6-4dd4-be4b-c9e6bb8baa95 LogTimestamp=2021-05-28T00:09:23.677040Z Entering 2021-05-28T00:09:23.677Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=95015b86-17f9-4111-ad24-ebc1677ac083 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:23.676936Z InvocationID=60b96a36-9eb6-4dd4-be4b-c9e6bb8baa95 LogTimestamp=2021-05-28T00:09:23.677040Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:09:23.775Z [qtp580718781-17] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:09:23.678Z InvocationID=ed43c0c0-1e9c-4331-a341-15d097a378c7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@7721eef2', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 96ms 2021-05-28T00:09:23.777Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=99 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:09:23.678Z InvocationID=ed43c0c0-1e9c-4331-a341-15d097a378c7 ErrorCode=400 LogTimestamp=2021-05-28T00:09:23.777205Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:09:23.778Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:09:23.865Z [qtp580718781-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:23.865742Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:23.865Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:23.865742Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:23.866Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:09:23.976Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=32020b96-5da0-4bb4-8df9-d860172a4eb6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:23.976440Z InvocationID=0966a58b-4456-42d3-b517-81fb73675fd3 LogTimestamp=2021-05-28T00:09:23.976561Z Entering 2021-05-28T00:09:23.977Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=32020b96-5da0-4bb4-8df9-d860172a4eb6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:23.976440Z InvocationID=0966a58b-4456-42d3-b517-81fb73675fd3 LogTimestamp=2021-05-28T00:09:23.976561Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:09:24.075Z [qtp580718781-15] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:09:23.977Z InvocationID=aa678c8c-c8fc-4a34-aa06-eb30ad2200d9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@7721eef2', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 97ms 2021-05-28T00:09:24.077Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=99 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:09:23.977Z InvocationID=aa678c8c-c8fc-4a34-aa06-eb30ad2200d9 ErrorCode=409 LogTimestamp=2021-05-28T00:09:24.076952Z ResponseDesc=Conflict Exiting. 2021-05-28T00:09:24.078Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:09:24.264Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:24.264465Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:24.265Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:24.264465Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:24.266Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:09:24.373Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=23a0e279-1651-4150-9b46-be313e6c456b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:24.373326Z InvocationID=dd2e5dfb-1c62-4a74-818a-83e09f0f4b7a LogTimestamp=2021-05-28T00:09:24.373532Z Entering 2021-05-28T00:09:24.374Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=23a0e279-1651-4150-9b46-be313e6c456b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:24.373326Z InvocationID=dd2e5dfb-1c62-4a74-818a-83e09f0f4b7a LogTimestamp=2021-05-28T00:09:24.373532Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:09:24.569Z [qtp580718781-96] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:09:24.374Z InvocationID=0872de15-6464-4983-a0f6-60382de7066f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@7721eef2', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 193ms 2021-05-28T00:09:24.570Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=196 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:09:24.374Z InvocationID=0872de15-6464-4983-a0f6-60382de7066f ErrorCode=409 LogTimestamp=2021-05-28T00:09:24.570045Z ResponseDesc=Conflict Exiting. 2021-05-28T00:09:24.570Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:09:24.674Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:24.673966Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:24.674Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:24.673966Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:24.674Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:09:24.776Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=54312824-6738-4a48-a537-45d48b80a162 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:24.776492Z InvocationID=09d25bb1-c0b8-4e32-9908-b4313e414c03 LogTimestamp=2021-05-28T00:09:24.776581Z Entering 2021-05-28T00:09:24.777Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=54312824-6738-4a48-a537-45d48b80a162 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:24.776492Z InvocationID=09d25bb1-c0b8-4e32-9908-b4313e414c03 LogTimestamp=2021-05-28T00:09:24.776581Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:09:24.876Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:09:24.777Z InvocationID=4aeb7770-f459-4c7e-90bc-ae5e87ba9049 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@7721eef2', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 98ms 2021-05-28T00:09:24.877Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=100 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:09:24.777Z InvocationID=4aeb7770-f459-4c7e-90bc-ae5e87ba9049 ErrorCode=409 LogTimestamp=2021-05-28T00:09:24.877152Z ResponseDesc=Conflict Exiting. 2021-05-28T00:09:24.878Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:09:25.267Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0de86af0-5925-4d92-ab65-e49628479026 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:25.267377Z InvocationID=2f1f8568-1ac3-455f-80e0-c87d03eeea36 LogTimestamp=2021-05-28T00:09:25.267497Z Entering 2021-05-28T00:09:25.267Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0de86af0-5925-4d92-ab65-e49628479026 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:25.267377Z InvocationID=2f1f8568-1ac3-455f-80e0-c87d03eeea36 LogTimestamp=2021-05-28T00:09:25.267497Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:09:25.377Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=108 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:09:25.268Z InvocationID=76d4ae9b-c7e2-45ac-b2cb-624560331fd2 LogTimestamp=2021-05-28T00:09:25.376851Z ResponseDesc=OK Exiting. 2021-05-28T00:09:25.377Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:09:25.465Z [qtp580718781-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:25.464970Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:25.465Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:25.464970Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:25.465Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:09:25.568Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5675b216-1899-42ba-ae2d-e777373497b7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServerIPAddress=10.242.102.221 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:25.568100Z InvocationID=d671eedd-7220-48ba-b8fe-e7a9822e274d LogTimestamp=2021-05-28T00:09:25.568192Z Entering 2021-05-28T00:09:25.568Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5675b216-1899-42ba-ae2d-e777373497b7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServerIPAddress=10.242.102.221 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:25.568100Z InvocationID=d671eedd-7220-48ba-b8fe-e7a9822e274d LogTimestamp=2021-05-28T00:09:25.568192Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:25.682Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:25.569Z InvocationID=d8c378de-197e-4bdd-a489-62de80a2e0f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:09:25.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a659a626-9809-47d1-8a17-077d6dc67c06 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:25.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a659a626-9809-47d1-8a17-077d6dc67c06 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7be13610 2021-05-28T00:09:25.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a659a626-9809-47d1-8a17-077d6dc67c06 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:25.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a659a626-9809-47d1-8a17-077d6dc67c06 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:28.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11f9fce1-a0f7-4d10-b4ff-a4f828c5e38b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:28.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11f9fce1-a0f7-4d10-b4ff-a4f828c5e38b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5fda304c 2021-05-28T00:09:28.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11f9fce1-a0f7-4d10-b4ff-a4f828c5e38b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:28.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11f9fce1-a0f7-4d10-b4ff-a4f828c5e38b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:30.856Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=5287 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:25.569Z InvocationID=d8c378de-197e-4bdd-a489-62de80a2e0f6 LogTimestamp=2021-05-28T00:09:30.856413Z ResponseDesc=Created Exiting. 2021-05-28T00:09:30.857Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:30.877Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:30.877489Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:30.877Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:30.877489Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:30.878Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:09:31.063Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a25a8f37-409e-468d-9e9c-db353469fdf1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:31.062615Z InvocationID=5246828f-854b-4f52-bdf3-5a50e06729a4 LogTimestamp=2021-05-28T00:09:31.062756Z Entering 2021-05-28T00:09:31.063Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a25a8f37-409e-468d-9e9c-db353469fdf1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:31.062615Z InvocationID=5246828f-854b-4f52-bdf3-5a50e06729a4 LogTimestamp=2021-05-28T00:09:31.062756Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:31.173Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:31.064Z InvocationID=f3681fb2-3917-46fb-9ccf-28c904733e02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:09:31.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40928167-f121-4108-8f28-37b06bc0f39a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:31.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40928167-f121-4108-8f28-37b06bc0f39a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ddce31f 2021-05-28T00:09:31.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40928167-f121-4108-8f28-37b06bc0f39a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:31.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40928167-f121-4108-8f28-37b06bc0f39a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:34.670Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3605 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:31.064Z InvocationID=f3681fb2-3917-46fb-9ccf-28c904733e02 LogTimestamp=2021-05-28T00:09:34.669905Z ResponseDesc=Created Exiting. 2021-05-28T00:09:34.670Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:34.776Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:34.776307Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:34.776Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:34.776307Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:34.778Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:09:34.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9496aae9-9ff5-4f40-bb53-aac1bfab66cb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:34.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9496aae9-9ff5-4f40-bb53-aac1bfab66cb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@220b5195 2021-05-28T00:09:34.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9496aae9-9ff5-4f40-bb53-aac1bfab66cb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:34.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9496aae9-9ff5-4f40-bb53-aac1bfab66cb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:34.877Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=563308d1-c4cc-471c-a52c-0505336f3e7f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:34.876917Z InvocationID=40ff0713-6fc9-41c0-8f22-73d645e2c286 LogTimestamp=2021-05-28T00:09:34.877080Z Entering 2021-05-28T00:09:34.877Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=563308d1-c4cc-471c-a52c-0505336f3e7f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:34.876917Z InvocationID=40ff0713-6fc9-41c0-8f22-73d645e2c286 LogTimestamp=2021-05-28T00:09:34.877080Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:34.963Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:34.878Z InvocationID=5f99e70d-3199-48fc-8f05-0f7267abc767 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:09:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:09:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:09:36.965Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2087 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:34.878Z InvocationID=5f99e70d-3199-48fc-8f05-0f7267abc767 LogTimestamp=2021-05-28T00:09:36.965104Z ResponseDesc=Created Exiting. 2021-05-28T00:09:36.965Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:37.062Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:37.062375Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:37.062Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:37.062375Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:37.063Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:09:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:09:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:09:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:09:37.272Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=93e7524c-810d-42da-a3b8-7219bf9e019e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:37.272091Z InvocationID=8591b03f-1c19-487b-85b2-7b4e72f1e85d LogTimestamp=2021-05-28T00:09:37.272247Z Entering 2021-05-28T00:09:37.272Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=93e7524c-810d-42da-a3b8-7219bf9e019e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:37.272091Z InvocationID=8591b03f-1c19-487b-85b2-7b4e72f1e85d LogTimestamp=2021-05-28T00:09:37.272247Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:37.368Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:37.273Z InvocationID=53729ab1-3485-45ac-886b-f158f6d2bda4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:09:37.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc3613b1-8570-4cff-8a70-d13ce16d5784 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:37.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc3613b1-8570-4cff-8a70-d13ce16d5784 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5028b272 2021-05-28T00:09:37.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc3613b1-8570-4cff-8a70-d13ce16d5784 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:37.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc3613b1-8570-4cff-8a70-d13ce16d5784 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:40.672Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3398 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:37.273Z InvocationID=53729ab1-3485-45ac-886b-f158f6d2bda4 LogTimestamp=2021-05-28T00:09:40.671635Z ResponseDesc=Created Exiting. 2021-05-28T00:09:40.672Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:40.774Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:40.774841Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:40.775Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:40.774841Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:40.775Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:09:40.875Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=250b0791-1569-4ed4-a7de-20d012fd6aa2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:40.875593Z InvocationID=6d3edb20-2b16-4988-a5aa-9f61722d1267 LogTimestamp=2021-05-28T00:09:40.875678Z Entering 2021-05-28T00:09:40.876Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=250b0791-1569-4ed4-a7de-20d012fd6aa2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:40.875593Z InvocationID=6d3edb20-2b16-4988-a5aa-9f61722d1267 LogTimestamp=2021-05-28T00:09:40.875678Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5945b98-e046-4b15-ac5c-af3486f32af2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5945b98-e046-4b15-ac5c-af3486f32af2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5efeb06b 2021-05-28T00:09:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5945b98-e046-4b15-ac5c-af3486f32af2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5945b98-e046-4b15-ac5c-af3486f32af2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:40.974Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:40.876Z InvocationID=d2735c31-15eb-417d-89c4-30c9619a2cab PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:09:43.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a87944-81c2-45e6-9676-fd0465f8d301 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:43.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a87944-81c2-45e6-9676-fd0465f8d301 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14f074b0 2021-05-28T00:09:43.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a87944-81c2-45e6-9676-fd0465f8d301 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:43.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a87944-81c2-45e6-9676-fd0465f8d301 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:43.877Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3000 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:40.876Z InvocationID=d2735c31-15eb-417d-89c4-30c9619a2cab LogTimestamp=2021-05-28T00:09:43.876567Z ResponseDesc=Created Exiting. 2021-05-28T00:09:43.877Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:43.977Z [qtp580718781-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:43.977138Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:43.977Z [qtp580718781-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:43.977138Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:43.978Z [qtp580718781-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:09:44.073Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9863f695-4fec-4eb1-9508-8fbb4ebf6d0f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:44.073485Z InvocationID=5cb3d110-b973-4ea8-9f0d-e25fda7f6d27 LogTimestamp=2021-05-28T00:09:44.073601Z Entering 2021-05-28T00:09:44.074Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9863f695-4fec-4eb1-9508-8fbb4ebf6d0f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:44.073485Z InvocationID=5cb3d110-b973-4ea8-9f0d-e25fda7f6d27 LogTimestamp=2021-05-28T00:09:44.073601Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:44.172Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:44.074Z InvocationID=10d71665-7926-47b5-b912-429d7b0b0fe0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:09:46.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=787ef679-f3f2-40a1-a7e5-b1e0ac2ef0d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:46.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=787ef679-f3f2-40a1-a7e5-b1e0ac2ef0d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c0f0842 2021-05-28T00:09:46.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=787ef679-f3f2-40a1-a7e5-b1e0ac2ef0d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:46.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=787ef679-f3f2-40a1-a7e5-b1e0ac2ef0d5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:47.066Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2992 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:44.074Z InvocationID=10d71665-7926-47b5-b912-429d7b0b0fe0 LogTimestamp=2021-05-28T00:09:47.066105Z ResponseDesc=Created Exiting. 2021-05-28T00:09:47.066Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:47.168Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:47.168252Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:47.168Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:47.168252Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:47.169Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:09:47.265Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=df857507-3994-4cf1-8e41-6f2916b68681 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:47.265087Z InvocationID=f872ad26-291c-4b1d-a882-1b2f6d6dadb1 LogTimestamp=2021-05-28T00:09:47.265209Z Entering 2021-05-28T00:09:47.265Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=df857507-3994-4cf1-8e41-6f2916b68681 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:47.265087Z InvocationID=f872ad26-291c-4b1d-a882-1b2f6d6dadb1 LogTimestamp=2021-05-28T00:09:47.265209Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:47.375Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:47.266Z InvocationID=b0ab0072-7aa9-401c-bc8e-1e9bbb8dac17 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:09:49.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009aa251-d306-4ef9-a160-0657ead30951 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:49.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009aa251-d306-4ef9-a160-0657ead30951 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79a19c93 2021-05-28T00:09:49.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009aa251-d306-4ef9-a160-0657ead30951 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:49.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009aa251-d306-4ef9-a160-0657ead30951 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:49.883Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2616 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:47.266Z InvocationID=b0ab0072-7aa9-401c-bc8e-1e9bbb8dac17 LogTimestamp=2021-05-28T00:09:49.882803Z ResponseDesc=Created Exiting. 2021-05-28T00:09:49.883Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:49.972Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:49.972758Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:49.973Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:49.972758Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:49.973Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:09:50.079Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=06b01197-9187-465b-93f9-5ccd35e3bdc9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:50.078558Z InvocationID=1ada0416-40e3-4800-b871-2e281077cfbf LogTimestamp=2021-05-28T00:09:50.078647Z Entering 2021-05-28T00:09:50.079Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=06b01197-9187-465b-93f9-5ccd35e3bdc9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:50.078558Z InvocationID=1ada0416-40e3-4800-b871-2e281077cfbf LogTimestamp=2021-05-28T00:09:50.078647Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:50.269Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:50.079Z InvocationID=63de58be-faa2-4d41-833b-6e74973a3769 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:09:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:09:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:09:52.121Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:09:52.174Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2095 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:50.079Z InvocationID=63de58be-faa2-4d41-833b-6e74973a3769 LogTimestamp=2021-05-28T00:09:52.174555Z ResponseDesc=Created Exiting. 2021-05-28T00:09:52.175Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:52.273Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:52.273597Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:52.274Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:52.273597Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:52.274Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:09:52.372Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1955c1e6-88fd-4a74-b8c6-96327cd70eb4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:52.372501Z InvocationID=093d8db0-6cb3-4bb3-8f5a-0cbc0f95a6ab LogTimestamp=2021-05-28T00:09:52.372583Z Entering 2021-05-28T00:09:52.373Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1955c1e6-88fd-4a74-b8c6-96327cd70eb4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:52.372501Z InvocationID=093d8db0-6cb3-4bb3-8f5a-0cbc0f95a6ab LogTimestamp=2021-05-28T00:09:52.372583Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:52.469Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:52.373Z InvocationID=fcb54d97-8e0c-4c86-9b80-293f3fb01f2e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:09:52.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e839ce7a-03f9-4ce4-aa3e-c0776fbd3be0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:52.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e839ce7a-03f9-4ce4-aa3e-c0776fbd3be0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52f70624 2021-05-28T00:09:52.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e839ce7a-03f9-4ce4-aa3e-c0776fbd3be0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:52.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e839ce7a-03f9-4ce4-aa3e-c0776fbd3be0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:55.456Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3083 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:52.373Z InvocationID=fcb54d97-8e0c-4c86-9b80-293f3fb01f2e LogTimestamp=2021-05-28T00:09:55.456281Z ResponseDesc=Created Exiting. 2021-05-28T00:09:55.457Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:55.473Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:55.473789Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:55.474Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:55.473789Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:55.474Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:09:55.576Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cd289621-1900-4518-99a7-82bfb9442ab0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:55.576053Z InvocationID=8200432c-b25c-4eaf-89a1-b205ed5fdae3 LogTimestamp=2021-05-28T00:09:55.576135Z Entering 2021-05-28T00:09:55.576Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cd289621-1900-4518-99a7-82bfb9442ab0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:55.576053Z InvocationID=8200432c-b25c-4eaf-89a1-b205ed5fdae3 LogTimestamp=2021-05-28T00:09:55.576135Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:55.671Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:55.576Z InvocationID=c984e9d4-a2f0-4cc0-8b4c-8273595fec26 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:09:55.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=392510c3-1f35-413e-9b56-6177c1d0aac8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:55.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=392510c3-1f35-413e-9b56-6177c1d0aac8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4535ad3e 2021-05-28T00:09:55.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=392510c3-1f35-413e-9b56-6177c1d0aac8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:55.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=392510c3-1f35-413e-9b56-6177c1d0aac8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:09:57.777Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2200 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:55.576Z InvocationID=c984e9d4-a2f0-4cc0-8b4c-8273595fec26 LogTimestamp=2021-05-28T00:09:57.776744Z ResponseDesc=Created Exiting. 2021-05-28T00:09:57.777Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:09:57.794Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:57.793966Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:09:57.794Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:09:57.793966Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:09:57.794Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:09:57.892Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6b7b8483-7b30-4efe-a054-98a813a56f6b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:57.892484Z InvocationID=ec176870-84e2-4cdb-97b6-0b396f7d02a0 LogTimestamp=2021-05-28T00:09:57.892560Z Entering 2021-05-28T00:09:57.893Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6b7b8483-7b30-4efe-a054-98a813a56f6b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:09:57.892484Z InvocationID=ec176870-84e2-4cdb-97b6-0b396f7d02a0 LogTimestamp=2021-05-28T00:09:57.892560Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:09:57.983Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:57.893Z InvocationID=65506bb1-eba6-4704-bbc3-286bd1a678a4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:09:59.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5fdc73-dc0b-4990-a096-4e4185a0daca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:09:59.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5fdc73-dc0b-4990-a096-4e4185a0daca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@516ce0c2 2021-05-28T00:09:59.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5fdc73-dc0b-4990-a096-4e4185a0daca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:09:59.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5fdc73-dc0b-4990-a096-4e4185a0daca PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:00.987Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3094 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:09:57.893Z InvocationID=65506bb1-eba6-4704-bbc3-286bd1a678a4 LogTimestamp=2021-05-28T00:10:00.987018Z ResponseDesc=Created Exiting. 2021-05-28T00:10:00.988Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:10:01.076Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:01.076829Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:10:01.077Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:01.076829Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:01.078Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:10:01.174Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a3207a3a-8eac-45e5-97c9-2710b5e33d4b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:01.173791Z InvocationID=d838cd22-a26e-4fcc-afc6-81356a5dde37 LogTimestamp=2021-05-28T00:10:01.173865Z Entering 2021-05-28T00:10:01.174Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a3207a3a-8eac-45e5-97c9-2710b5e33d4b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:01.173791Z InvocationID=d838cd22-a26e-4fcc-afc6-81356a5dde37 LogTimestamp=2021-05-28T00:10:01.173865Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:10:01.478Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:01.175Z InvocationID=df79f336-e87c-4686-8049-e90af07f83e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:10:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0ad18de-2136-4da3-ba12-0bdc042af1f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0ad18de-2136-4da3-ba12-0bdc042af1f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@245cbb78 2021-05-28T00:10:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0ad18de-2136-4da3-ba12-0bdc042af1f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0ad18de-2136-4da3-ba12-0bdc042af1f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:04.483Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3307 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:01.175Z InvocationID=df79f336-e87c-4686-8049-e90af07f83e6 LogTimestamp=2021-05-28T00:10:04.482831Z ResponseDesc=Created Exiting. 2021-05-28T00:10:04.483Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:10:04.772Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:04.772834Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:10:04.773Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:04.772834Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:04.773Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:10:04.880Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e5ed35e4-784d-4a24-9632-67b3eb5b1984 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:04.880117Z InvocationID=70d82d53-1b1b-4bb6-b361-7dc2d0bf866d LogTimestamp=2021-05-28T00:10:04.880220Z Entering 2021-05-28T00:10:04.880Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e5ed35e4-784d-4a24-9632-67b3eb5b1984 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:04.880117Z InvocationID=70d82d53-1b1b-4bb6-b361-7dc2d0bf866d LogTimestamp=2021-05-28T00:10:04.880220Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:10:04.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fab47986-41cb-4267-b73b-2935718dd183 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:04.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fab47986-41cb-4267-b73b-2935718dd183 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e704ff1 2021-05-28T00:10:04.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fab47986-41cb-4267-b73b-2935718dd183 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:04.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fab47986-41cb-4267-b73b-2935718dd183 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:04.973Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:04.881Z InvocationID=7380005a-3856-4240-a68f-1b4e981d1846 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:10:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:10:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:10:07.368Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2487 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:04.881Z InvocationID=7380005a-3856-4240-a68f-1b4e981d1846 LogTimestamp=2021-05-28T00:10:07.368221Z ResponseDesc=Created Exiting. 2021-05-28T00:10:07.369Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:10:07.481Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:07.481335Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:10:07.481Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:07.481335Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:07.482Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:10:07.581Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9d240d93-ffed-4db6-80cf-4f04f2dd0b68 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:07.581551Z InvocationID=bf7fbbc9-7238-4a79-88b3-818af169a28a LogTimestamp=2021-05-28T00:10:07.581625Z Entering 2021-05-28T00:10:07.582Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9d240d93-ffed-4db6-80cf-4f04f2dd0b68 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:07.581551Z InvocationID=bf7fbbc9-7238-4a79-88b3-818af169a28a LogTimestamp=2021-05-28T00:10:07.581625Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:10:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=063164ff-afd2-45e8-8c6c-c784f8a320bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=063164ff-afd2-45e8-8c6c-c784f8a320bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2644475c 2021-05-28T00:10:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=063164ff-afd2-45e8-8c6c-c784f8a320bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=063164ff-afd2-45e8-8c6c-c784f8a320bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:07.679Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:07.582Z InvocationID=f329c707-c5d6-4e94-aa06-636c51c86495 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:10:09.863Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2281 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:07.582Z InvocationID=f329c707-c5d6-4e94-aa06-636c51c86495 LogTimestamp=2021-05-28T00:10:09.863624Z ResponseDesc=Created Exiting. 2021-05-28T00:10:09.864Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:10:09.988Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:09.988675Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:10:09.989Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:10:09.988675Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:09.989Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:10:10.676Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1a7acf2a-49de-423d-9b65-84e20218e9c9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:10.675760Z InvocationID=bf6244bd-9594-4350-bebc-bbf85cbbd474 LogTimestamp=2021-05-28T00:10:10.675870Z Entering 2021-05-28T00:10:10.676Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1a7acf2a-49de-423d-9b65-84e20218e9c9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:10.675760Z InvocationID=bf6244bd-9594-4350-bebc-bbf85cbbd474 LogTimestamp=2021-05-28T00:10:10.675870Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:10:10.779Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:10.676Z InvocationID=d4435232-45ae-484c-8315-3c6e1a6a3a77 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:10:11.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32d0cd69-c72c-4303-bc2f-7eadcc20e3be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:11.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32d0cd69-c72c-4303-bc2f-7eadcc20e3be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1098d9a6 2021-05-28T00:10:11.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32d0cd69-c72c-4303-bc2f-7eadcc20e3be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:11.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32d0cd69-c72c-4303-bc2f-7eadcc20e3be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:13.370Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2693 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:10:10.676Z InvocationID=d4435232-45ae-484c-8315-3c6e1a6a3a77 LogTimestamp=2021-05-28T00:10:13.369780Z ResponseDesc=Created Exiting. 2021-05-28T00:10:13.370Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:10:13.781Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e786c4c2-0794-41b0-9885-11161eb24c7f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:13.781054Z InvocationID=aa89d10c-cd37-42d9-9ccf-40fe78df231d LogTimestamp=2021-05-28T00:10:13.781196Z Entering 2021-05-28T00:10:13.781Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e786c4c2-0794-41b0-9885-11161eb24c7f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:13.781054Z InvocationID=aa89d10c-cd37-42d9-9ccf-40fe78df231d LogTimestamp=2021-05-28T00:10:13.781196Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:10:13.881Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=99 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:10:13.782Z InvocationID=f8d6c905-d8e2-428b-8969-790d547d1bad LogTimestamp=2021-05-28T00:10:13.881179Z ResponseDesc=OK Exiting. 2021-05-28T00:10:13.881Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:10:14.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5e0e72-0299-48a2-87c5-447f55e27afb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:14.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5e0e72-0299-48a2-87c5-447f55e27afb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@515e37b6 2021-05-28T00:10:14.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5e0e72-0299-48a2-87c5-447f55e27afb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:14.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5e0e72-0299-48a2-87c5-447f55e27afb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:14.483Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b6d742cb-9c1c-4cdb-be49-7196044fb571 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:14.483026Z InvocationID=e7c06b8c-2696-4aa7-96b3-da0cb3490101 LogTimestamp=2021-05-28T00:10:14.483130Z Entering 2021-05-28T00:10:14.483Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b6d742cb-9c1c-4cdb-be49-7196044fb571 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:14.483026Z InvocationID=e7c06b8c-2696-4aa7-96b3-da0cb3490101 LogTimestamp=2021-05-28T00:10:14.483130Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:10:17.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1affafa2-83bc-4341-a65e-b6c911e5f274 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:17.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1affafa2-83bc-4341-a65e-b6c911e5f274 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d9de5ab 2021-05-28T00:10:17.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1affafa2-83bc-4341-a65e-b6c911e5f274 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:17.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1affafa2-83bc-4341-a65e-b6c911e5f274 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:17.673Z [qtp580718781-19] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:10:14.483Z InvocationID=77bd2ea5-2b51-4b7a-ad75-7f0b885e820d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 3116388492 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n3116388492 100% \n 2021-05-28T00:10:17.723Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=3239 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:10:14.483Z InvocationID=77bd2ea5-2b51-4b7a-ad75-7f0b885e820d LogTimestamp=2021-05-28T00:10:17.722993Z ResponseDesc=OK Exiting. 2021-05-28T00:10:17.723Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:10:18.082Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=af445203-1056-4226-95b7-9cf513889654 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:18.082041Z InvocationID=aa173d7e-af1d-4a3e-9878-01f8bd81ed25 LogTimestamp=2021-05-28T00:10:18.082182Z Entering 2021-05-28T00:10:18.082Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=af445203-1056-4226-95b7-9cf513889654 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:18.082041Z InvocationID=aa173d7e-af1d-4a3e-9878-01f8bd81ed25 LogTimestamp=2021-05-28T00:10:18.082182Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:10:18.473Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:10:18.082Z InvocationID=cbbfd798-d4ce-4c65-bf46-0bcf526788d0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:10:18.559Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:18.082Z InvocationID=cbbfd798-d4ce-4c65-bf46-0bcf526788d0 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:10:18.560Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:18.082Z InvocationID=cbbfd798-d4ce-4c65-bf46-0bcf526788d0 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:10:18.562Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=479 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:10:18.082Z InvocationID=cbbfd798-d4ce-4c65-bf46-0bcf526788d0 ErrorCode=500 LogTimestamp=2021-05-28T00:10:18.561778Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:18.562Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:10:20.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2cf49d-8788-4114-839f-c755fb8c4a09 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:20.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2cf49d-8788-4114-839f-c755fb8c4a09 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19e73fe3 2021-05-28T00:10:20.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2cf49d-8788-4114-839f-c755fb8c4a09 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:20.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2cf49d-8788-4114-839f-c755fb8c4a09 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:10:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:10:23.360Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3df2efe-a397-43ea-877b-d9cd44406efa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:23.360Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3df2efe-a397-43ea-877b-d9cd44406efa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fe48090 2021-05-28T00:10:23.360Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3df2efe-a397-43ea-877b-d9cd44406efa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:23.360Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3df2efe-a397-43ea-877b-d9cd44406efa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:28.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d4eecd1-a882-4436-99e2-5386e2dcc509 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:28.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d4eecd1-a882-4436-99e2-5386e2dcc509 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f27ed5d 2021-05-28T00:10:28.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d4eecd1-a882-4436-99e2-5386e2dcc509 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:28.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d4eecd1-a882-4436-99e2-5386e2dcc509 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:28.677Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=81efc299-0045-4b60-a81e-e54e2aaf09df InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:28.676819Z InvocationID=ef52d4d2-8c19-43a1-a62d-0002f192e367 LogTimestamp=2021-05-28T00:10:28.676946Z Entering 2021-05-28T00:10:28.677Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=81efc299-0045-4b60-a81e-e54e2aaf09df InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:28.676819Z InvocationID=ef52d4d2-8c19-43a1-a62d-0002f192e367 LogTimestamp=2021-05-28T00:10:28.676946Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:10:28.834Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:10:28.677Z InvocationID=a646ad4c-8b27-4483-b2db-d06033705ef3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:10:29.174Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:28.677Z InvocationID=a646ad4c-8b27-4483-b2db-d06033705ef3 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:10:29.175Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:28.677Z InvocationID=a646ad4c-8b27-4483-b2db-d06033705ef3 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:10:29.176Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=498 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:10:28.677Z InvocationID=a646ad4c-8b27-4483-b2db-d06033705ef3 ErrorCode=500 LogTimestamp=2021-05-28T00:10:29.175678Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:29.176Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:10:29.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24fd0da3-0ff7-47a5-bef1-9c9b306ac3d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:29.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24fd0da3-0ff7-47a5-bef1-9c9b306ac3d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d65edaf 2021-05-28T00:10:29.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24fd0da3-0ff7-47a5-bef1-9c9b306ac3d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:29.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24fd0da3-0ff7-47a5-bef1-9c9b306ac3d4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:32.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac99658a-db79-477e-a819-08465492fc2a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:32.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac99658a-db79-477e-a819-08465492fc2a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d5dc1c7 2021-05-28T00:10:32.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac99658a-db79-477e-a819-08465492fc2a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:32.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac99658a-db79-477e-a819-08465492fc2a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:35.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:10:35.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:10:36.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6abb21bb-90a3-4243-acc5-28c94b2943bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:36.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6abb21bb-90a3-4243-acc5-28c94b2943bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56d636fc 2021-05-28T00:10:36.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6abb21bb-90a3-4243-acc5-28c94b2943bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:36.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6abb21bb-90a3-4243-acc5-28c94b2943bc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:10:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:10:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:10:38.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f55dc54-49ef-4fce-964c-31838507f97c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:38.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f55dc54-49ef-4fce-964c-31838507f97c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cb647e 2021-05-28T00:10:38.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f55dc54-49ef-4fce-964c-31838507f97c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:38.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f55dc54-49ef-4fce-964c-31838507f97c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:39.283Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=84157300-dff1-408a-99cd-96e4290c35e4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:39.282760Z InvocationID=85bb674d-2893-4ff6-99cc-ecec191e63c3 LogTimestamp=2021-05-28T00:10:39.282847Z Entering 2021-05-28T00:10:39.283Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=84157300-dff1-408a-99cd-96e4290c35e4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:39.282760Z InvocationID=85bb674d-2893-4ff6-99cc-ecec191e63c3 LogTimestamp=2021-05-28T00:10:39.282847Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:10:39.434Z [qtp580718781-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:10:39.283Z InvocationID=8ba6b2c7-b425-4f6b-8da2-26b5b47e0939 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:10:39.471Z [qtp580718781-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:39.283Z InvocationID=8ba6b2c7-b425-4f6b-8da2-26b5b47e0939 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:10:39.471Z [qtp580718781-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:39.283Z InvocationID=8ba6b2c7-b425-4f6b-8da2-26b5b47e0939 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:10:39.472Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=189 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:10:39.283Z InvocationID=8ba6b2c7-b425-4f6b-8da2-26b5b47e0939 ErrorCode=500 LogTimestamp=2021-05-28T00:10:39.472373Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:39.473Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:10:41.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=748db103-7a7c-4937-963b-e1f603ac5695 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:41.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=748db103-7a7c-4937-963b-e1f603ac5695 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3471e99d 2021-05-28T00:10:41.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=748db103-7a7c-4937-963b-e1f603ac5695 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:41.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=748db103-7a7c-4937-963b-e1f603ac5695 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:46.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5814813-8123-4a8c-992b-566a83cffc90 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:46.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5814813-8123-4a8c-992b-566a83cffc90 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6771452d 2021-05-28T00:10:46.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5814813-8123-4a8c-992b-566a83cffc90 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:46.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5814813-8123-4a8c-992b-566a83cffc90 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:47.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f116d06c-56f1-453e-9f46-8826f60117cc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:47.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f116d06c-56f1-453e-9f46-8826f60117cc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77fee9 2021-05-28T00:10:47.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f116d06c-56f1-453e-9f46-8826f60117cc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:47.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f116d06c-56f1-453e-9f46-8826f60117cc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:49.674Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5f8e39ea-3d2b-45dc-8ea6-7b74bd88bdd1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:49.673684Z InvocationID=261369aa-5ca4-4faf-baf3-2aefc6b5f84a LogTimestamp=2021-05-28T00:10:49.673809Z Entering 2021-05-28T00:10:49.674Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5f8e39ea-3d2b-45dc-8ea6-7b74bd88bdd1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:49.673684Z InvocationID=261369aa-5ca4-4faf-baf3-2aefc6b5f84a LogTimestamp=2021-05-28T00:10:49.673809Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:10:49.827Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:10:49.674Z InvocationID=5a3ee61f-090c-4539-9a9c-d30a7a01e751 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:10:49.873Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:49.674Z InvocationID=5a3ee61f-090c-4539-9a9c-d30a7a01e751 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:10:49.873Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:49.674Z InvocationID=5a3ee61f-090c-4539-9a9c-d30a7a01e751 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:10:49.875Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=200 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:10:49.674Z InvocationID=5a3ee61f-090c-4539-9a9c-d30a7a01e751 ErrorCode=500 LogTimestamp=2021-05-28T00:10:49.874605Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:10:49.875Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:10:50.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acb3c331-61c1-4d2b-9e43-5b4db6bbcb17 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:50.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acb3c331-61c1-4d2b-9e43-5b4db6bbcb17 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d42820d 2021-05-28T00:10:50.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acb3c331-61c1-4d2b-9e43-5b4db6bbcb17 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:50.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acb3c331-61c1-4d2b-9e43-5b4db6bbcb17 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:10:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:10:52.123Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:10:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21e764ab-18d5-4e73-aae5-3be65a09942c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21e764ab-18d5-4e73-aae5-3be65a09942c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66c4a539 2021-05-28T00:10:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21e764ab-18d5-4e73-aae5-3be65a09942c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21e764ab-18d5-4e73-aae5-3be65a09942c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:55.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9a3bf43-53d8-4419-8b42-4d1f64220f01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:55.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9a3bf43-53d8-4419-8b42-4d1f64220f01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b71dce1 2021-05-28T00:10:55.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9a3bf43-53d8-4419-8b42-4d1f64220f01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:55.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9a3bf43-53d8-4419-8b42-4d1f64220f01 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:58.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=440f24a6-7e6c-4a44-b9de-ff4ff2814cf3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:10:58.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=440f24a6-7e6c-4a44-b9de-ff4ff2814cf3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@344efbd0 2021-05-28T00:10:58.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=440f24a6-7e6c-4a44-b9de-ff4ff2814cf3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:10:58.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=440f24a6-7e6c-4a44-b9de-ff4ff2814cf3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:10:59.980Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f8cf35f9-6d68-42a8-aed2-a27c30c25625 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:59.979633Z InvocationID=e51d8a4f-d9fc-4eea-9070-596d0cc72cd4 LogTimestamp=2021-05-28T00:10:59.979782Z Entering 2021-05-28T00:10:59.980Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f8cf35f9-6d68-42a8-aed2-a27c30c25625 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:10:59.979633Z InvocationID=e51d8a4f-d9fc-4eea-9070-596d0cc72cd4 LogTimestamp=2021-05-28T00:10:59.979782Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:11:00.144Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:10:59.980Z InvocationID=e15a9916-e744-4f2c-9b26-f5936cb102cb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:11:00.174Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:59.980Z InvocationID=e15a9916-e744-4f2c-9b26-f5936cb102cb ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:11:00.174Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:10:59.980Z InvocationID=e15a9916-e744-4f2c-9b26-f5936cb102cb ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:11:00.175Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=195 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:10:59.980Z InvocationID=e15a9916-e744-4f2c-9b26-f5936cb102cb ErrorCode=500 LogTimestamp=2021-05-28T00:11:00.175382Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:11:00.176Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:11:02.181Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad4b6530-0161-40ec-8b8e-eed959c43344 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:02.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad4b6530-0161-40ec-8b8e-eed959c43344 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7dd545be 2021-05-28T00:11:02.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad4b6530-0161-40ec-8b8e-eed959c43344 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:02.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad4b6530-0161-40ec-8b8e-eed959c43344 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dee922a-6df6-466c-9ade-ff065e69dc3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dee922a-6df6-466c-9ade-ff065e69dc3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e7bda50 2021-05-28T00:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dee922a-6df6-466c-9ade-ff065e69dc3b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dee922a-6df6-466c-9ade-ff065e69dc3b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:11:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:11:08.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d26d1f03-8aa2-4c1c-8844-48af422b7347 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:08.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d26d1f03-8aa2-4c1c-8844-48af422b7347 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3be88faa 2021-05-28T00:11:08.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d26d1f03-8aa2-4c1c-8844-48af422b7347 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:08.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d26d1f03-8aa2-4c1c-8844-48af422b7347 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:10.471Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a06f8eb9-ed31-4f79-9470-177d668709af InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:10.471166Z InvocationID=b5a586d4-a51c-4185-9d5d-3a6de6d99ec8 LogTimestamp=2021-05-28T00:11:10.471331Z Entering 2021-05-28T00:11:10.471Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a06f8eb9-ed31-4f79-9470-177d668709af InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:10.471166Z InvocationID=b5a586d4-a51c-4185-9d5d-3a6de6d99ec8 LogTimestamp=2021-05-28T00:11:10.471331Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:11:10.582Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:11:10.472Z InvocationID=bed7c6b7-e059-4979-87f7-198be5125755 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:11:10.770Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:10.472Z InvocationID=bed7c6b7-e059-4979-87f7-198be5125755 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:11:10.779Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:10.472Z InvocationID=bed7c6b7-e059-4979-87f7-198be5125755 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:11:10.780Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=308 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:11:10.472Z InvocationID=bed7c6b7-e059-4979-87f7-198be5125755 ErrorCode=500 LogTimestamp=2021-05-28T00:11:10.780158Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:11:10.780Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:11:10.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afd9558-3200-442b-817b-2a5fffe6a65c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:10.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afd9558-3200-442b-817b-2a5fffe6a65c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@783b737c 2021-05-28T00:11:10.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afd9558-3200-442b-817b-2a5fffe6a65c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:10.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afd9558-3200-442b-817b-2a5fffe6a65c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:14.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c5c6ea-675b-4073-8e72-f023e37ed481 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:14.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c5c6ea-675b-4073-8e72-f023e37ed481 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f942170 2021-05-28T00:11:14.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c5c6ea-675b-4073-8e72-f023e37ed481 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:14.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c5c6ea-675b-4073-8e72-f023e37ed481 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:19.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=892595ef-dc37-42a1-a44e-b614925ee141 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:19.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=892595ef-dc37-42a1-a44e-b614925ee141 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f5dc788 2021-05-28T00:11:19.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=892595ef-dc37-42a1-a44e-b614925ee141 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:19.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=892595ef-dc37-42a1-a44e-b614925ee141 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:11:20.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:11:20.961Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=423e0e39-d0ca-48ac-a5af-00572e09560b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:20.960698Z InvocationID=da7c73e7-becd-49a0-ad33-d4b97bc0d9c1 LogTimestamp=2021-05-28T00:11:20.960861Z Entering 2021-05-28T00:11:20.961Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=423e0e39-d0ca-48ac-a5af-00572e09560b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:20.960698Z InvocationID=da7c73e7-becd-49a0-ad33-d4b97bc0d9c1 LogTimestamp=2021-05-28T00:11:20.960861Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:11:21.135Z [qtp580718781-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:11:20.961Z InvocationID=dc6534f2-f562-48ed-996e-283734086a86 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:11:21.174Z [qtp580718781-17] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:20.961Z InvocationID=dc6534f2-f562-48ed-996e-283734086a86 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:11:21.175Z [qtp580718781-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:20.961Z InvocationID=dc6534f2-f562-48ed-996e-283734086a86 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:11:21.175Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=214 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:11:20.961Z InvocationID=dc6534f2-f562-48ed-996e-283734086a86 ErrorCode=500 LogTimestamp=2021-05-28T00:11:21.175617Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:11:21.176Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:11:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d14d32a-cbc5-42b3-abd4-3c7b130f7577 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d14d32a-cbc5-42b3-abd4-3c7b130f7577 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30379416 2021-05-28T00:11:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d14d32a-cbc5-42b3-abd4-3c7b130f7577 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d14d32a-cbc5-42b3-abd4-3c7b130f7577 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03cef376-831c-4a8b-bb75-38cd33580d7f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03cef376-831c-4a8b-bb75-38cd33580d7f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13b4a1ff 2021-05-28T00:11:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03cef376-831c-4a8b-bb75-38cd33580d7f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03cef376-831c-4a8b-bb75-38cd33580d7f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7169587c-f8a5-48d0-b058-817fa476653c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7169587c-f8a5-48d0-b058-817fa476653c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d259d58 2021-05-28T00:11:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7169587c-f8a5-48d0-b058-817fa476653c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7169587c-f8a5-48d0-b058-817fa476653c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:29.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafc3800-7cf6-4ae3-853c-3d0b7cefaef2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:29.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafc3800-7cf6-4ae3-853c-3d0b7cefaef2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c6578f3 2021-05-28T00:11:29.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafc3800-7cf6-4ae3-853c-3d0b7cefaef2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:29.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fafc3800-7cf6-4ae3-853c-3d0b7cefaef2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:31.375Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=04fc6a0d-5194-4376-955a-a28c48a41186 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:31.375304Z InvocationID=6e543922-472a-4ff5-976f-c89828c8c43c LogTimestamp=2021-05-28T00:11:31.375412Z Entering 2021-05-28T00:11:31.376Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=04fc6a0d-5194-4376-955a-a28c48a41186 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:31.375304Z InvocationID=6e543922-472a-4ff5-976f-c89828c8c43c LogTimestamp=2021-05-28T00:11:31.375412Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:11:31.485Z [qtp580718781-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:11:31.376Z InvocationID=f9e59bd1-6528-46a9-897a-f62d0636e488 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:11:31.668Z [qtp580718781-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:31.376Z InvocationID=f9e59bd1-6528-46a9-897a-f62d0636e488 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:11:31.668Z [qtp580718781-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:31.376Z InvocationID=f9e59bd1-6528-46a9-897a-f62d0636e488 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:11:31.670Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=293 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:11:31.376Z InvocationID=f9e59bd1-6528-46a9-897a-f62d0636e488 ErrorCode=500 LogTimestamp=2021-05-28T00:11:31.669589Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:11:31.670Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:11:33.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dea0e1ec-1b8e-4867-9bae-7d8c90e6b69d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:33.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dea0e1ec-1b8e-4867-9bae-7d8c90e6b69d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@307ecc4d 2021-05-28T00:11:33.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dea0e1ec-1b8e-4867-9bae-7d8c90e6b69d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:33.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dea0e1ec-1b8e-4867-9bae-7d8c90e6b69d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:11:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:11:36.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1da90b79-5e05-409e-a276-92a6b1cd9442 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:36.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1da90b79-5e05-409e-a276-92a6b1cd9442 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67338e4 2021-05-28T00:11:36.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1da90b79-5e05-409e-a276-92a6b1cd9442 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:36.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1da90b79-5e05-409e-a276-92a6b1cd9442 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:11:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:11:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:11:39.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4b7a2ec-e0f0-4405-9c38-a7bb4338bfa4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:39.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4b7a2ec-e0f0-4405-9c38-a7bb4338bfa4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@378daa2f 2021-05-28T00:11:39.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4b7a2ec-e0f0-4405-9c38-a7bb4338bfa4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:39.282Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4b7a2ec-e0f0-4405-9c38-a7bb4338bfa4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:39.730Z [ComponentsCleanThread-1] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null lock resource with name DELETE_LOCKER 2021-05-28T00:11:42.474Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f1587ed3-d769-4a72-a35b-cfd2da706024 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:42.474515Z InvocationID=7d6ba44f-1dd4-4ce9-83b8-863042b6cfbf LogTimestamp=2021-05-28T00:11:42.474678Z Entering 2021-05-28T00:11:42.475Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f1587ed3-d769-4a72-a35b-cfd2da706024 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:42.474515Z InvocationID=7d6ba44f-1dd4-4ce9-83b8-863042b6cfbf LogTimestamp=2021-05-28T00:11:42.474678Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:11:42.585Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:11:42.475Z InvocationID=2d5c30fd-51f2-44d6-b501-944caff53815 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:11:43.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ec98991-57bd-4eac-9b82-8443b8592858 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:43.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ec98991-57bd-4eac-9b82-8443b8592858 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2038eecd 2021-05-28T00:11:43.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ec98991-57bd-4eac-9b82-8443b8592858 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:43.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ec98991-57bd-4eac-9b82-8443b8592858 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:43.170Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:42.475Z InvocationID=2d5c30fd-51f2-44d6-b501-944caff53815 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:11:43.170Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:42.475Z InvocationID=2d5c30fd-51f2-44d6-b501-944caff53815 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:11:43.171Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=696 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:11:42.475Z InvocationID=2d5c30fd-51f2-44d6-b501-944caff53815 ErrorCode=500 LogTimestamp=2021-05-28T00:11:43.171559Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:11:43.172Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:11:43.711Z [ComponentsCleanThread-1] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null Temporary storage failure while checking locks org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.getSliceWithRetries(ConsistentKeyLocker.java:464) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:426) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:175) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:154) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:72) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:279) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1265) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1249) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponentByName(GraphLockOperation.java:87) at org.openecomp.sdc.be.components.scheduledtasks.ComponentsCleanBusinessLogic.lockDeleteOperation(ComponentsCleanBusinessLogic.java:131) at org.openecomp.sdc.be.components.scheduledtasks.ComponentsCleanBusinessLogic.isDeleteOperationLockFailed(ComponentsCleanBusinessLogic.java:143) at org.openecomp.sdc.be.components.scheduledtasks.ComponentsCleanBusinessLogic.cleanComponents(ComponentsCleanBusinessLogic.java:89) at org.openecomp.sdc.be.components.scheduledtasks.ComponentsCleanBusinessLogic.cleanComponents(ComponentsCleanBusinessLogic.java:77) at org.openecomp.sdc.be.components.scheduledtasks.AsdcComponentsCleanerTask.run(AsdcComponentsCleanerTask.java:136) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:11:44.668Z [ComponentsCleanThread-1] INFO o.o.s.b.c.s.ComponentsCleanBusinessLogic RequestID=null Lock cleanup operation is done with result = OK 2021-05-28T00:11:45.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=134ea66e-9978-4238-a607-937843f2b752 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:45.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=134ea66e-9978-4238-a607-937843f2b752 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66625223 2021-05-28T00:11:45.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=134ea66e-9978-4238-a607-937843f2b752 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:45.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=134ea66e-9978-4238-a607-937843f2b752 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:48.482Z [ComponentsCleanThread-1] INFO o.o.s.b.c.s.ComponentsCleanBusinessLogic RequestID=null Unlock cleanup operation is done with result = OK 2021-05-28T00:11:48.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b70ae691-9cc6-4eb4-9a64-96faf5974e19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:48.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b70ae691-9cc6-4eb4-9a64-96faf5974e19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7949f213 2021-05-28T00:11:48.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b70ae691-9cc6-4eb4-9a64-96faf5974e19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:48.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b70ae691-9cc6-4eb4-9a64-96faf5974e19 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:49.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0b1867-3059-4ce7-87ed-cc9b6ae70678 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:49.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0b1867-3059-4ce7-87ed-cc9b6ae70678 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23cfe3c7 2021-05-28T00:11:49.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0b1867-3059-4ce7-87ed-cc9b6ae70678 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:49.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0b1867-3059-4ce7-87ed-cc9b6ae70678 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:11:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:11:52.123Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:11:53.383Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=32c6aeb2-5899-4773-a5a8-d68768b6732b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:53.382752Z InvocationID=e803abe4-4341-4ea4-bd67-f4aa075aa8e1 LogTimestamp=2021-05-28T00:11:53.382890Z Entering 2021-05-28T00:11:53.383Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=32c6aeb2-5899-4773-a5a8-d68768b6732b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:11:53.382752Z InvocationID=e803abe4-4341-4ea4-bd67-f4aa075aa8e1 LogTimestamp=2021-05-28T00:11:53.382890Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:11:53.519Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:11:53.383Z InvocationID=a5ee82d8-4386-4c7d-ae5a-a8b8873fd971 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:11:53.667Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:53.383Z InvocationID=a5ee82d8-4386-4c7d-ae5a-a8b8873fd971 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:11:53.668Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:11:53.383Z InvocationID=a5ee82d8-4386-4c7d-ae5a-a8b8873fd971 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:11:53.669Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=285 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:11:53.383Z InvocationID=a5ee82d8-4386-4c7d-ae5a-a8b8873fd971 ErrorCode=500 LogTimestamp=2021-05-28T00:11:53.668858Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:11:53.669Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:11:54.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71310ece-d6fc-4966-ad0d-7e680afdf5cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:54.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71310ece-d6fc-4966-ad0d-7e680afdf5cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44663ff5 2021-05-28T00:11:54.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71310ece-d6fc-4966-ad0d-7e680afdf5cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:54.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71310ece-d6fc-4966-ad0d-7e680afdf5cf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:55.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9be8b2c-f175-455b-914b-39a4609bc17b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:55.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9be8b2c-f175-455b-914b-39a4609bc17b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30e18e10 2021-05-28T00:11:55.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9be8b2c-f175-455b-914b-39a4609bc17b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:55.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9be8b2c-f175-455b-914b-39a4609bc17b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:11:59.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7558ec99-5a68-43a7-8795-b15b19c0c1bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:11:59.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7558ec99-5a68-43a7-8795-b15b19c0c1bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68e3b75 2021-05-28T00:11:59.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7558ec99-5a68-43a7-8795-b15b19c0c1bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:11:59.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7558ec99-5a68-43a7-8795-b15b19c0c1bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:02.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f9450e7-a55a-4371-be8b-b7d6d68c231b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:02.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f9450e7-a55a-4371-be8b-b7d6d68c231b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a8321b8 2021-05-28T00:12:02.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f9450e7-a55a-4371-be8b-b7d6d68c231b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:02.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f9450e7-a55a-4371-be8b-b7d6d68c231b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:03.875Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a1f946b3-6db5-40d0-bd2d-155b88a82120 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS RemoteHost=10.242.170.174 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:12:03.874773Z InvocationID=0d316d74-f2c8-453b-84cc-9eade83b38cd LogTimestamp=2021-05-28T00:12:03.874982Z Entering 2021-05-28T00:12:03.875Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a1f946b3-6db5-40d0-bd2d-155b88a82120 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.174 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.174 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:12:03.874773Z InvocationID=0d316d74-f2c8-453b-84cc-9eade83b38cd LogTimestamp=2021-05-28T00:12:03.874982Z localAddr=10.242.102.221 remoteAddr=10.242.170.174 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:12:04.039Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:12:03.875Z InvocationID=61165f09-fef1-4ba7-af5b-ccea895c39b4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:12:04.071Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:12:03.875Z InvocationID=61165f09-fef1-4ba7-af5b-ccea895c39b4 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:12:04.071Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:12:03.875Z InvocationID=61165f09-fef1-4ba7-af5b-ccea895c39b4 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:12:04.073Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=197 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:12:03.875Z InvocationID=61165f09-fef1-4ba7-af5b-ccea895c39b4 ErrorCode=500 LogTimestamp=2021-05-28T00:12:04.072678Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:12:04.073Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:12:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4255456c-15e0-4dda-959a-b08094ef234b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4255456c-15e0-4dda-959a-b08094ef234b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f6f248a 2021-05-28T00:12:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4255456c-15e0-4dda-959a-b08094ef234b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4255456c-15e0-4dda-959a-b08094ef234b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:04.873Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=4255456c-15e0-4dda-959a-b08094ef234b PartnerName=SDC.BE I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {s}->https://sdc-onboarding-be.onap:8445: The target server failed to respond 2021-05-28T00:12:04.873Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=4255456c-15e0-4dda-959a-b08094ef234b PartnerName=SDC.BE Retrying request to {s}->https://sdc-onboarding-be.onap:8445 2021-05-28T00:12:04.881Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4255456c-15e0-4dda-959a-b08094ef234b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:12:04.881Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4255456c-15e0-4dda-959a-b08094ef234b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:12:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:12:07.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fa346ac-fb88-4e39-83e0-85b4416a161b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:07.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fa346ac-fb88-4e39-83e0-85b4416a161b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22b67c92 2021-05-28T00:12:07.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fa346ac-fb88-4e39-83e0-85b4416a161b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:07.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fa346ac-fb88-4e39-83e0-85b4416a161b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:07.585Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0fa346ac-fb88-4e39-83e0-85b4416a161b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:12:07.586Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0fa346ac-fb88-4e39-83e0-85b4416a161b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:10.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb7cb5e-4a84-4117-abf0-857ce4afa5ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:10.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb7cb5e-4a84-4117-abf0-857ce4afa5ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40413a05 2021-05-28T00:12:10.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb7cb5e-4a84-4117-abf0-857ce4afa5ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:10.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb7cb5e-4a84-4117-abf0-857ce4afa5ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:10.977Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0cb7cb5e-4a84-4117-abf0-857ce4afa5ea ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:12:10.978Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0cb7cb5e-4a84-4117-abf0-857ce4afa5ea ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:14.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b29f7b8-0ad1-444c-a097-5593240931b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:14.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b29f7b8-0ad1-444c-a097-5593240931b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8f490b2 2021-05-28T00:12:14.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b29f7b8-0ad1-444c-a097-5593240931b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:14.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b29f7b8-0ad1-444c-a097-5593240931b9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:14.062Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6b29f7b8-0ad1-444c-a097-5593240931b9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:12:14.063Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6b29f7b8-0ad1-444c-a097-5593240931b9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:17.023Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bfee44c-7286-406e-ae2e-4e2e55851a17 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:17.023Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bfee44c-7286-406e-ae2e-4e2e55851a17 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41ae0b7e 2021-05-28T00:12:17.023Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bfee44c-7286-406e-ae2e-4e2e55851a17 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:17.023Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bfee44c-7286-406e-ae2e-4e2e55851a17 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:17.026Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2bfee44c-7286-406e-ae2e-4e2e55851a17 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:12:17.027Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2bfee44c-7286-406e-ae2e-4e2e55851a17 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:19.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cc8e666-2464-4d3c-9dd4-02047eb056ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:19.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cc8e666-2464-4d3c-9dd4-02047eb056ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c67b749 2021-05-28T00:12:19.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cc8e666-2464-4d3c-9dd4-02047eb056ff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:19.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cc8e666-2464-4d3c-9dd4-02047eb056ff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:19.782Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6cc8e666-2464-4d3c-9dd4-02047eb056ff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:12:19.782Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6cc8e666-2464-4d3c-9dd4-02047eb056ff ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:12:20.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:12:22.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18d5e55-289e-45c0-8b13-3511952b0313 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:22.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18d5e55-289e-45c0-8b13-3511952b0313 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b1d080e 2021-05-28T00:12:22.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18d5e55-289e-45c0-8b13-3511952b0313 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:22.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18d5e55-289e-45c0-8b13-3511952b0313 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:22.965Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f18d5e55-289e-45c0-8b13-3511952b0313 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:12:22.965Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f18d5e55-289e-45c0-8b13-3511952b0313 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39f01642-3d79-445e-83d7-3eac76ec969b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39f01642-3d79-445e-83d7-3eac76ec969b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d7707d0 2021-05-28T00:12:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39f01642-3d79-445e-83d7-3eac76ec969b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39f01642-3d79-445e-83d7-3eac76ec969b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:29.270Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=39f01642-3d79-445e-83d7-3eac76ec969b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:12:29.270Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=39f01642-3d79-445e-83d7-3eac76ec969b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:30.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69128d50-791c-4810-8c67-4776dd7041c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:30.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69128d50-791c-4810-8c67-4776dd7041c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e41c30f 2021-05-28T00:12:30.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69128d50-791c-4810-8c67-4776dd7041c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:30.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69128d50-791c-4810-8c67-4776dd7041c3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:33.086Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=69128d50-791c-4810-8c67-4776dd7041c3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:12:33.086Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=69128d50-791c-4810-8c67-4776dd7041c3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:33.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0785f81-cd41-47f1-bfe6-c03ce2d09149 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:33.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0785f81-cd41-47f1-bfe6-c03ce2d09149 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4da5c75c 2021-05-28T00:12:33.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0785f81-cd41-47f1-bfe6-c03ce2d09149 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:33.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0785f81-cd41-47f1-bfe6-c03ce2d09149 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:12:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:12:36.980Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e0785f81-cd41-47f1-bfe6-c03ce2d09149 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:12:36.981Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e0785f81-cd41-47f1-bfe6-c03ce2d09149 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:12:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:12:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:12:37.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fb638e8-d0cb-43a4-9a72-d9b9057930cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:37.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fb638e8-d0cb-43a4-9a72-d9b9057930cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b5dacd 2021-05-28T00:12:37.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fb638e8-d0cb-43a4-9a72-d9b9057930cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:37.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fb638e8-d0cb-43a4-9a72-d9b9057930cf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:40.483Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1fb638e8-d0cb-43a4-9a72-d9b9057930cf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:12:40.483Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1fb638e8-d0cb-43a4-9a72-d9b9057930cf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:41.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82601882-33f2-4991-9d82-6fff45b02c5d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:41.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82601882-33f2-4991-9d82-6fff45b02c5d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d1c3edb 2021-05-28T00:12:41.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82601882-33f2-4991-9d82-6fff45b02c5d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:41.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82601882-33f2-4991-9d82-6fff45b02c5d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:44.579Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=82601882-33f2-4991-9d82-6fff45b02c5d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:12:44.579Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=82601882-33f2-4991-9d82-6fff45b02c5d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:45.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0846a601-8064-4c4c-be05-b239429e3404 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:45.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0846a601-8064-4c4c-be05-b239429e3404 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@728f54e3 2021-05-28T00:12:45.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0846a601-8064-4c4c-be05-b239429e3404 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:45.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0846a601-8064-4c4c-be05-b239429e3404 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:48.487Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0846a601-8064-4c4c-be05-b239429e3404 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:12:48.487Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0846a601-8064-4c4c-be05-b239429e3404 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:49.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cbd314b4-77bb-4acb-b4e2-377c147c17d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:49.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cbd314b4-77bb-4acb-b4e2-377c147c17d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74f82507 2021-05-28T00:12:49.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cbd314b4-77bb-4acb-b4e2-377c147c17d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:49.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cbd314b4-77bb-4acb-b4e2-377c147c17d3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:12:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:12:52.132Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:12:52.878Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=cbd314b4-77bb-4acb-b4e2-377c147c17d3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:12:52.878Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=cbd314b4-77bb-4acb-b4e2-377c147c17d3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:12:54.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae89fb14-af5c-435e-ad2b-ab60f80881bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:12:54.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae89fb14-af5c-435e-ad2b-ab60f80881bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3422541d 2021-05-28T00:12:54.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae89fb14-af5c-435e-ad2b-ab60f80881bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:12:54.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae89fb14-af5c-435e-ad2b-ab60f80881bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:12:59.560Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ae89fb14-af5c-435e-ad2b-ab60f80881bd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:12:59.560Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ae89fb14-af5c-435e-ad2b-ab60f80881bd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:13:00.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e95b2af-f263-48cb-aecf-55f1c8e815bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:00.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e95b2af-f263-48cb-aecf-55f1c8e815bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4821893c 2021-05-28T00:13:00.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e95b2af-f263-48cb-aecf-55f1c8e815bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:00.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e95b2af-f263-48cb-aecf-55f1c8e815bf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:13:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:13:03.819Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0e95b2af-f263-48cb-aecf-55f1c8e815bf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:13:03.819Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0e95b2af-f263-48cb-aecf-55f1c8e815bf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:13:04.479Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7be7b68d-b595-4a73-ba3c-afd716b94134 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:04.478347Z InvocationID=483bb677-10cb-49df-aadd-1c73da40e5df LogTimestamp=2021-05-28T00:13:04.478620Z Entering 2021-05-28T00:13:04.479Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7be7b68d-b595-4a73-ba3c-afd716b94134 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:04.478347Z InvocationID=483bb677-10cb-49df-aadd-1c73da40e5df LogTimestamp=2021-05-28T00:13:04.478620Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:13:04.481Z [qtp580718781-19] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:13:04.480Z InvocationID=dfcc6f85-8f45-463b-8140-d97e88da485b ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:13:04.515Z [qtp580718781-19] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=3 EndTimestamp=2021-05-28T00:13:04.483Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.175 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:13:04.480Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=dfcc6f85-8f45-463b-8140-d97e88da485b ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@1052a867 2021-05-28T00:13:04.515Z [qtp580718781-19] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=3 EndTimestamp=2021-05-28T00:13:04.483Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.175 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:13:04.480Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=dfcc6f85-8f45-463b-8140-d97e88da485b ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@1052a867 2021-05-28T00:13:04.517Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:04.517656Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:04.517Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:04.517656Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:04.518Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:13:05.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:13:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:13:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:13:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8837db7-9637-4ec4-a572-f458cf6aaf1d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8837db7-9637-4ec4-a572-f458cf6aaf1d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4313aeba 2021-05-28T00:13:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8837db7-9637-4ec4-a572-f458cf6aaf1d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8837db7-9637-4ec4-a572-f458cf6aaf1d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:07.673Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:13:07.996Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d8837db7-9637-4ec4-a572-f458cf6aaf1d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:13:07.996Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=d8837db7-9637-4ec4-a572-f458cf6aaf1d ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:13:08.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f49ff3d7-ad50-411a-a5e5-84057d15112c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:08.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f49ff3d7-ad50-411a-a5e5-84057d15112c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@397110b2 2021-05-28T00:13:08.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f49ff3d7-ad50-411a-a5e5-84057d15112c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:08.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f49ff3d7-ad50-411a-a5e5-84057d15112c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:10.979Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f49ff3d7-ad50-411a-a5e5-84057d15112c ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:13:10.980Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f49ff3d7-ad50-411a-a5e5-84057d15112c ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:13:12.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e5754af-dffe-496c-a03f-2f892339d5f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:12.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e5754af-dffe-496c-a03f-2f892339d5f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@371c46f5 2021-05-28T00:13:12.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e5754af-dffe-496c-a03f-2f892339d5f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:12.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e5754af-dffe-496c-a03f-2f892339d5f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:14.763Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d4b9a700-59e8-4a7f-b30a-b4312ffa36f2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:14.762829Z InvocationID=98a2599f-1de8-49d0-9d76-1c7eaa26f20b LogTimestamp=2021-05-28T00:13:14.762930Z Entering 2021-05-28T00:13:14.763Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d4b9a700-59e8-4a7f-b30a-b4312ffa36f2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:14.762829Z InvocationID=98a2599f-1de8-49d0-9d76-1c7eaa26f20b LogTimestamp=2021-05-28T00:13:14.762930Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:13:14.969Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=206 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:13:14.763Z InvocationID=32e78bdf-a5e7-4e33-b33e-b13b6e6cc338 LogTimestamp=2021-05-28T00:13:14.969067Z ResponseDesc=OK Exiting. 2021-05-28T00:13:14.969Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:13:15.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52bcffe2-7545-45eb-b646-f0fe132ba03b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:15.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52bcffe2-7545-45eb-b646-f0fe132ba03b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43abac37 2021-05-28T00:13:15.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52bcffe2-7545-45eb-b646-f0fe132ba03b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:15.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52bcffe2-7545-45eb-b646-f0fe132ba03b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:15.188Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:15.188332Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:15.188Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:15.188332Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:15.188Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:13:15.367Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6ec727bc-d8a1-42b7-873f-a652a5b5e29a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:15.366661Z InvocationID=69bbd1b5-9c0d-4534-b196-377cffb35e76 LogTimestamp=2021-05-28T00:13:15.366765Z Entering 2021-05-28T00:13:15.367Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6ec727bc-d8a1-42b7-873f-a652a5b5e29a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:15.366661Z InvocationID=69bbd1b5-9c0d-4534-b196-377cffb35e76 LogTimestamp=2021-05-28T00:13:15.366765Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:13:15.767Z [qtp580718781-11] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:13:15.367Z InvocationID=cdef8dbd-ea3a-4d38-8897-7a33920c51fd PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@719d54c5', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 398ms 2021-05-28T00:13:15.768Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=401 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:13:15.367Z InvocationID=cdef8dbd-ea3a-4d38-8897-7a33920c51fd ErrorCode=409 LogTimestamp=2021-05-28T00:13:15.768541Z ResponseDesc=Conflict Exiting. 2021-05-28T00:13:15.769Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:13:15.782Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:15.782458Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:15.782Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:15.782458Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:15.783Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:13:15.962Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5e0617d6-b92e-4ad6-a383-c964ae3aa3a8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:15.961826Z InvocationID=ad274ff8-0e19-4d12-8d7c-f27cba1bf34d LogTimestamp=2021-05-28T00:13:15.961934Z Entering 2021-05-28T00:13:15.962Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5e0617d6-b92e-4ad6-a383-c964ae3aa3a8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:15.961826Z InvocationID=ad274ff8-0e19-4d12-8d7c-f27cba1bf34d LogTimestamp=2021-05-28T00:13:15.961934Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:13:16.075Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:13:15.962Z InvocationID=3f2ab126-c79a-4a88-b67b-1a4bac1098ab PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@719d54c5', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 111ms 2021-05-28T00:13:16.076Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=114 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:13:15.962Z InvocationID=3f2ab126-c79a-4a88-b67b-1a4bac1098ab ErrorCode=400 LogTimestamp=2021-05-28T00:13:16.076195Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:13:16.077Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:13:16.171Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=9520 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:13:06.650Z InvocationID=604e1e2c-7104-48b7-86c0-6c00344b5896 ErrorCode=500 LogTimestamp=2021-05-28T00:13:16.170993Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:16.171Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:13:16.261Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6b03229e-6836-41f2-823c-6f305c0ea611 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:16.261534Z InvocationID=d9eadcc5-306c-4a5e-b4ea-c25ba05ac446 LogTimestamp=2021-05-28T00:13:16.261609Z Entering 2021-05-28T00:13:16.261Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6b03229e-6836-41f2-823c-6f305c0ea611 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:16.261534Z InvocationID=d9eadcc5-306c-4a5e-b4ea-c25ba05ac446 LogTimestamp=2021-05-28T00:13:16.261609Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:13:16.372Z [qtp580718781-17] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:13:16.262Z InvocationID=cfaf7c07-f030-4b43-bae2-979e16ea02f2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@719d54c5', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 109ms 2021-05-28T00:13:16.373Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=111 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:13:16.262Z InvocationID=cfaf7c07-f030-4b43-bae2-979e16ea02f2 ErrorCode=400 LogTimestamp=2021-05-28T00:13:16.373087Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:13:16.373Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:13:16.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f096506-8290-4fab-855e-e2ec63c50304 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:16.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f096506-8290-4fab-855e-e2ec63c50304 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f3fc9c5 2021-05-28T00:13:16.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f096506-8290-4fab-855e-e2ec63c50304 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:16.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f096506-8290-4fab-855e-e2ec63c50304 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:16.474Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=1645 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:13:14.828Z InvocationID=84896586-862b-4af1-a461-d51d8f5a2cff ErrorCode=500 LogTimestamp=2021-05-28T00:13:16.473778Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:16.474Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:13:16.686Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=082b5e9f-d4a7-44b2-9a77-70e71a0bb1db InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:16.685827Z InvocationID=def843df-e043-481b-9261-a0dacdc48b9b LogTimestamp=2021-05-28T00:13:16.685897Z Entering 2021-05-28T00:13:16.686Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=082b5e9f-d4a7-44b2-9a77-70e71a0bb1db InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:16.685827Z InvocationID=def843df-e043-481b-9261-a0dacdc48b9b LogTimestamp=2021-05-28T00:13:16.685897Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:13:16.858Z [qtp580718781-12] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:13:16.686Z InvocationID=3bfeedeb-1b39-42ba-9eb1-51ff761fd385 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@719d54c5', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 171ms 2021-05-28T00:13:16.860Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=174 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:13:16.686Z InvocationID=3bfeedeb-1b39-42ba-9eb1-51ff761fd385 ErrorCode=409 LogTimestamp=2021-05-28T00:13:16.860049Z ResponseDesc=Conflict Exiting. 2021-05-28T00:13:16.860Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:13:16.972Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:16.972869Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:16.973Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:16.972869Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:16.973Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:13:17.062Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fcb72d19-944c-4d02-aede-61468fd1110b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:17.061859Z InvocationID=72273d83-9d25-44b7-9e51-95ee73da2135 LogTimestamp=2021-05-28T00:13:17.061933Z Entering 2021-05-28T00:13:17.062Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fcb72d19-944c-4d02-aede-61468fd1110b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:17.061859Z InvocationID=72273d83-9d25-44b7-9e51-95ee73da2135 LogTimestamp=2021-05-28T00:13:17.061933Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:13:17.378Z [qtp580718781-18] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:13:17.062Z InvocationID=14be0208-a368-4224-a9ad-21ed6dec2054 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@719d54c5', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 314ms 2021-05-28T00:13:17.379Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=316 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:13:17.062Z InvocationID=14be0208-a368-4224-a9ad-21ed6dec2054 ErrorCode=409 LogTimestamp=2021-05-28T00:13:17.378969Z ResponseDesc=Conflict Exiting. 2021-05-28T00:13:17.379Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:13:17.473Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:17.473056Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:17.473Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:17.473056Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:17.473Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:13:17.662Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=510db27d-4cc5-4f9f-a8bd-0e02f6b650ca InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:17.662357Z InvocationID=770bff4d-ee49-451a-8737-b68576fe8f7c LogTimestamp=2021-05-28T00:13:17.662470Z Entering 2021-05-28T00:13:17.663Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=510db27d-4cc5-4f9f-a8bd-0e02f6b650ca InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:17.662357Z InvocationID=770bff4d-ee49-451a-8737-b68576fe8f7c LogTimestamp=2021-05-28T00:13:17.662470Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:13:17.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceab3db2-fe46-4227-b291-a357e376d5aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:17.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceab3db2-fe46-4227-b291-a357e376d5aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e7abe10 2021-05-28T00:13:17.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceab3db2-fe46-4227-b291-a357e376d5aa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:17.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceab3db2-fe46-4227-b291-a357e376d5aa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:17.882Z [qtp580718781-11] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:13:17.663Z InvocationID=67f4d53e-0e38-4c31-b2a0-7922b833d875 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@719d54c5', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 218ms 2021-05-28T00:13:17.885Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=222 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:13:17.663Z InvocationID=67f4d53e-0e38-4c31-b2a0-7922b833d875 ErrorCode=409 LogTimestamp=2021-05-28T00:13:17.885102Z ResponseDesc=Conflict Exiting. 2021-05-28T00:13:17.885Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:13:18.401Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=03297501-26ed-403a-b11c-93be464b80f2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:18.400907Z InvocationID=b5ee5aea-dee9-4f6a-9201-67beae9362d3 LogTimestamp=2021-05-28T00:13:18.400963Z Entering 2021-05-28T00:13:18.401Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=03297501-26ed-403a-b11c-93be464b80f2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:18.400907Z InvocationID=b5ee5aea-dee9-4f6a-9201-67beae9362d3 LogTimestamp=2021-05-28T00:13:18.400963Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:13:18.472Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=71 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:13:18.401Z InvocationID=f43850f1-22e9-47a1-a019-f9b99a76c7e4 LogTimestamp=2021-05-28T00:13:18.472306Z ResponseDesc=OK Exiting. 2021-05-28T00:13:18.472Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:13:18.758Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:18.758724Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:18.758Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:18.758724Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:18.759Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:13:18.779Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8c8c0850-5f26-4eef-a8ab-5d5573e129da InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:18.778926Z InvocationID=16b1a2eb-a505-4876-aca9-1e0189303132 LogTimestamp=2021-05-28T00:13:18.779021Z Entering 2021-05-28T00:13:18.780Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8c8c0850-5f26-4eef-a8ab-5d5573e129da InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:18.778926Z InvocationID=16b1a2eb-a505-4876-aca9-1e0189303132 LogTimestamp=2021-05-28T00:13:18.779021Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:18.869Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:18.780Z InvocationID=fba12c04-4f0b-44ce-a23f-f420b616dbbc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:13:19.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1c3673e-05a6-4551-a96a-11d57bce76b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:19.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1c3673e-05a6-4551-a96a-11d57bce76b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27df3975 2021-05-28T00:13:19.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1c3673e-05a6-4551-a96a-11d57bce76b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:19.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1c3673e-05a6-4551-a96a-11d57bce76b9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:13:20.518Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:13:20.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3e6e4d-5e0e-4a2b-85ea-440f2f05210b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:20.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3e6e4d-5e0e-4a2b-85ea-440f2f05210b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38ec1234 2021-05-28T00:13:20.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3e6e4d-5e0e-4a2b-85ea-440f2f05210b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:20.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a3e6e4d-5e0e-4a2b-85ea-440f2f05210b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:21.578Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2797 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:18.780Z InvocationID=fba12c04-4f0b-44ce-a23f-f420b616dbbc LogTimestamp=2021-05-28T00:13:21.577771Z ResponseDesc=Created Exiting. 2021-05-28T00:13:21.578Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:21.768Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:21.768295Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:21.768Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:21.768295Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:21.769Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:13:21.965Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=adf7f166-c22c-40bb-b7cf-ac13dd3f257d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:21.964656Z InvocationID=3e7c24f5-1c30-4a04-a255-23a8bc216ac1 LogTimestamp=2021-05-28T00:13:21.964716Z Entering 2021-05-28T00:13:21.965Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=adf7f166-c22c-40bb-b7cf-ac13dd3f257d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:21.964656Z InvocationID=3e7c24f5-1c30-4a04-a255-23a8bc216ac1 LogTimestamp=2021-05-28T00:13:21.964716Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:21.972Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:21.965Z InvocationID=3409140c-33ac-4de7-bf7e-0f91a4023f21 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:13:22.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=175e7dbb-d9f2-4467-a447-f1bcc3b1a4c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:22.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=175e7dbb-d9f2-4467-a447-f1bcc3b1a4c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6564ccdd 2021-05-28T00:13:22.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=175e7dbb-d9f2-4467-a447-f1bcc3b1a4c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:22.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=175e7dbb-d9f2-4467-a447-f1bcc3b1a4c4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:24.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f0f31f6-c5ae-45bc-adf1-5401ff69bb01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:24.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f0f31f6-c5ae-45bc-adf1-5401ff69bb01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b887abc 2021-05-28T00:13:24.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f0f31f6-c5ae-45bc-adf1-5401ff69bb01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:24.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2f0f31f6-c5ae-45bc-adf1-5401ff69bb01 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:24.664Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2698 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:21.965Z InvocationID=3409140c-33ac-4de7-bf7e-0f91a4023f21 LogTimestamp=2021-05-28T00:13:24.663164Z ResponseDesc=Created Exiting. 2021-05-28T00:13:24.665Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:24.858Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:24.858288Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:24.858Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:24.858288Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:24.859Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:13:24.985Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=61fd6136-d71e-47bc-96a8-fcd62c4a5d11 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:24.984677Z InvocationID=a2449c34-7fd8-42dc-9500-d495be1d351b LogTimestamp=2021-05-28T00:13:24.984766Z Entering 2021-05-28T00:13:24.985Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=61fd6136-d71e-47bc-96a8-fcd62c4a5d11 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:24.984677Z InvocationID=a2449c34-7fd8-42dc-9500-d495be1d351b LogTimestamp=2021-05-28T00:13:24.984766Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:25.075Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:24.986Z InvocationID=e5191ef4-c0dd-4811-b05d-bd7114b4393d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:13:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:13:29.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:13:29.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e782add6-2d57-427f-b28f-cf8ff64221d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:29.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e782add6-2d57-427f-b28f-cf8ff64221d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15c2e629 2021-05-28T00:13:29.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e782add6-2d57-427f-b28f-cf8ff64221d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:29.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e782add6-2d57-427f-b28f-cf8ff64221d0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:31.782Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:13:32.178Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=7192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:24.986Z InvocationID=e5191ef4-c0dd-4811-b05d-bd7114b4393d LogTimestamp=2021-05-28T00:13:32.178095Z ResponseDesc=Created Exiting. 2021-05-28T00:13:32.178Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:32.371Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:32.371505Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:32.371Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:32.371505Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:32.372Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:13:32.559Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6a5806c1-7acc-4491-b737-0500bc8ea433 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:32.559106Z InvocationID=80b90793-a878-4efc-b0a7-9945cd7fbc2a LogTimestamp=2021-05-28T00:13:32.559168Z Entering 2021-05-28T00:13:32.559Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6a5806c1-7acc-4491-b737-0500bc8ea433 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:32.559106Z InvocationID=80b90793-a878-4efc-b0a7-9945cd7fbc2a LogTimestamp=2021-05-28T00:13:32.559168Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:32.660Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:32.559Z InvocationID=6cda5376-ea49-4dc0-b773-d654a6da5d96 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:13:32.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16529946-f989-43ac-a421-4c7fecb50859 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:32.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16529946-f989-43ac-a421-4c7fecb50859 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4de73b17 2021-05-28T00:13:32.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16529946-f989-43ac-a421-4c7fecb50859 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:32.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16529946-f989-43ac-a421-4c7fecb50859 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:33.265Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=16529946-f989-43ac-a421-4c7fecb50859 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:13:33.265Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=16529946-f989-43ac-a421-4c7fecb50859 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:13:34.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98add681-b253-43af-9514-449b153012a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:34.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98add681-b253-43af-9514-449b153012a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ea60452 2021-05-28T00:13:34.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98add681-b253-43af-9514-449b153012a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:34.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98add681-b253-43af-9514-449b153012a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:34.377Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=98add681-b253-43af-9514-449b153012a9 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:13:34.378Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=98add681-b253-43af-9514-449b153012a9 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:13:34.476Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1917 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:32.559Z InvocationID=6cda5376-ea49-4dc0-b773-d654a6da5d96 LogTimestamp=2021-05-28T00:13:34.476063Z ResponseDesc=Created Exiting. 2021-05-28T00:13:34.476Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:34.571Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:34.571403Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:34.571Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:34.571403Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:34.572Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:13:34.668Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f9fd3437-3a9b-4d62-bcc9-912c18062399 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:34.668704Z InvocationID=11ef115c-94e8-4d0e-9032-413f04849652 LogTimestamp=2021-05-28T00:13:34.668765Z Entering 2021-05-28T00:13:34.669Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f9fd3437-3a9b-4d62-bcc9-912c18062399 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:34.668704Z InvocationID=11ef115c-94e8-4d0e-9032-413f04849652 LogTimestamp=2021-05-28T00:13:34.668765Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:34.678Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:34.669Z InvocationID=e8112ae2-d226-4af0-86e0-dbace01b2d7c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:13:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:13:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:13:35.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b707570-adb5-4d97-b7b2-5a96874086e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:35.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b707570-adb5-4d97-b7b2-5a96874086e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@673d2a0 2021-05-28T00:13:35.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b707570-adb5-4d97-b7b2-5a96874086e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:35.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b707570-adb5-4d97-b7b2-5a96874086e6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:36.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0349b3a3-dfc2-41e2-8b6c-da236265b8a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:36.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0349b3a3-dfc2-41e2-8b6c-da236265b8a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4028fe9 2021-05-28T00:13:36.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0349b3a3-dfc2-41e2-8b6c-da236265b8a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:36.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0349b3a3-dfc2-41e2-8b6c-da236265b8a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:13:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:13:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:13:37.760Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3090 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:34.669Z InvocationID=e8112ae2-d226-4af0-86e0-dbace01b2d7c LogTimestamp=2021-05-28T00:13:37.759697Z ResponseDesc=Created Exiting. 2021-05-28T00:13:37.760Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:37.861Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:37.861674Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:37.861Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:37.861674Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:37.862Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:13:38.077Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8d026069-361b-466e-bcc7-d80c7f48e9e2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:38.076922Z InvocationID=f2ff8b50-9870-408d-a560-4ff4de4feb6a LogTimestamp=2021-05-28T00:13:38.076985Z Entering 2021-05-28T00:13:38.077Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8d026069-361b-466e-bcc7-d80c7f48e9e2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:38.076922Z InvocationID=f2ff8b50-9870-408d-a560-4ff4de4feb6a LogTimestamp=2021-05-28T00:13:38.076985Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:38.168Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:38.077Z InvocationID=10003fc6-f0f5-43ea-9da3-67eba177499d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:13:38.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d6ff416-aee7-49e1-b132-14f4b13166da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:38.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d6ff416-aee7-49e1-b132-14f4b13166da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70c923fd 2021-05-28T00:13:38.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d6ff416-aee7-49e1-b132-14f4b13166da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:38.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d6ff416-aee7-49e1-b132-14f4b13166da PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:40.666Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2589 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:38.077Z InvocationID=10003fc6-f0f5-43ea-9da3-67eba177499d LogTimestamp=2021-05-28T00:13:40.666356Z ResponseDesc=Created Exiting. 2021-05-28T00:13:40.666Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:40.776Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:40.776440Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:40.776Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:40.776440Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:40.777Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:13:40.880Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=93793523-a2e5-4ed3-9f58-2d29318acad6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:40.880095Z InvocationID=883bd24a-9b31-479a-a7e7-2e8d331ee82d LogTimestamp=2021-05-28T00:13:40.880163Z Entering 2021-05-28T00:13:40.880Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=93793523-a2e5-4ed3-9f58-2d29318acad6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:40.880095Z InvocationID=883bd24a-9b31-479a-a7e7-2e8d331ee82d LogTimestamp=2021-05-28T00:13:40.880163Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:40.966Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:40.880Z InvocationID=f61b4239-11c5-47e6-8963-dfa0ffef83c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:13:40.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85ed4bd7-a454-46ea-8efd-372195f35ebd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:40.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85ed4bd7-a454-46ea-8efd-372195f35ebd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ef33d22 2021-05-28T00:13:40.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85ed4bd7-a454-46ea-8efd-372195f35ebd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:40.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85ed4bd7-a454-46ea-8efd-372195f35ebd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:42.868Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1988 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:40.880Z InvocationID=f61b4239-11c5-47e6-8963-dfa0ffef83c8 LogTimestamp=2021-05-28T00:13:42.868694Z ResponseDesc=Created Exiting. 2021-05-28T00:13:42.869Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:42.892Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:42.891949Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:42.892Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:42.891949Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:42.892Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:13:43.065Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1a1f099e-5a10-4eaf-bf8f-cb11960ce6af InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:43.064726Z InvocationID=1ce603dd-1a45-4954-80ca-bbf9b9a8afd0 LogTimestamp=2021-05-28T00:13:43.064784Z Entering 2021-05-28T00:13:43.065Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1a1f099e-5a10-4eaf-bf8f-cb11960ce6af InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:43.064726Z InvocationID=1ce603dd-1a45-4954-80ca-bbf9b9a8afd0 LogTimestamp=2021-05-28T00:13:43.064784Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:43.171Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:43.065Z InvocationID=4d1d49f6-7eb3-4a48-9d88-fb5a0bf92b36 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:13:44.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abadb37-06f1-4c8e-9853-6b5f861ef2ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:44.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abadb37-06f1-4c8e-9853-6b5f861ef2ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@568d108e 2021-05-28T00:13:44.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abadb37-06f1-4c8e-9853-6b5f861ef2ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:44.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2abadb37-06f1-4c8e-9853-6b5f861ef2ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:45.469Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2403 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:43.065Z InvocationID=4d1d49f6-7eb3-4a48-9d88-fb5a0bf92b36 LogTimestamp=2021-05-28T00:13:45.468925Z ResponseDesc=Created Exiting. 2021-05-28T00:13:45.469Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:45.571Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:45.571490Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:45.571Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:45.571490Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:45.572Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:13:45.673Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bcdc8601-92c1-40d2-a905-5f8be7a81dd1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:45.672842Z InvocationID=50bd3efc-82ec-4ff6-bd9f-fd28ae44e4c2 LogTimestamp=2021-05-28T00:13:45.672911Z Entering 2021-05-28T00:13:45.673Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bcdc8601-92c1-40d2-a905-5f8be7a81dd1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:45.672842Z InvocationID=50bd3efc-82ec-4ff6-bd9f-fd28ae44e4c2 LogTimestamp=2021-05-28T00:13:45.672911Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:45.771Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:45.673Z InvocationID=89f443a6-58c3-4db7-bb53-d442d5b34583 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:13:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49a7444f-6948-4309-b7a7-9e4aaa0b925a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49a7444f-6948-4309-b7a7-9e4aaa0b925a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4660a071 2021-05-28T00:13:46.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49a7444f-6948-4309-b7a7-9e4aaa0b925a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:46.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49a7444f-6948-4309-b7a7-9e4aaa0b925a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:48.361Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2687 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:45.673Z InvocationID=89f443a6-58c3-4db7-bb53-d442d5b34583 LogTimestamp=2021-05-28T00:13:48.360958Z ResponseDesc=Created Exiting. 2021-05-28T00:13:48.361Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:48.466Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:48.465962Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:48.466Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:48.465962Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:48.467Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:13:48.574Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ffee5c44-f700-499c-ba87-e85377280961 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:48.574487Z InvocationID=b0c95ccd-71ef-4f7f-a462-8df03e591dfb LogTimestamp=2021-05-28T00:13:48.574575Z Entering 2021-05-28T00:13:48.575Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ffee5c44-f700-499c-ba87-e85377280961 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:48.574487Z InvocationID=b0c95ccd-71ef-4f7f-a462-8df03e591dfb LogTimestamp=2021-05-28T00:13:48.574575Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:48.585Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:48.575Z InvocationID=8d91db06-0d81-4997-9b91-6b6e5190e0cc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:13:49.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aad4c22-c59b-4544-b867-40159bf8da46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:49.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aad4c22-c59b-4544-b867-40159bf8da46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f8fa58b 2021-05-28T00:13:49.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aad4c22-c59b-4544-b867-40159bf8da46 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:49.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aad4c22-c59b-4544-b867-40159bf8da46 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:13:50.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:13:51.766Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3191 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:48.575Z InvocationID=8d91db06-0d81-4997-9b91-6b6e5190e0cc LogTimestamp=2021-05-28T00:13:51.766551Z ResponseDesc=Created Exiting. 2021-05-28T00:13:51.767Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:51.782Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:51.782823Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:51.783Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:51.782823Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:51.783Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:13:51.881Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=09f56a32-3246-4cca-802a-5ae528204b23 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:51.880917Z InvocationID=0d9b6d1e-82e3-477a-8c12-59bcd41392f0 LogTimestamp=2021-05-28T00:13:51.880988Z Entering 2021-05-28T00:13:51.881Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=09f56a32-3246-4cca-802a-5ae528204b23 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:51.880917Z InvocationID=0d9b6d1e-82e3-477a-8c12-59bcd41392f0 LogTimestamp=2021-05-28T00:13:51.880988Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:51.981Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:51.881Z InvocationID=2d74ae9f-f0df-4c8d-bac2-04104fc15ac4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:13:52.146Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:13:52.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e29819cf-d405-4eb2-9973-f7f37f7d20d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:52.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e29819cf-d405-4eb2-9973-f7f37f7d20d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28c28d75 2021-05-28T00:13:52.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e29819cf-d405-4eb2-9973-f7f37f7d20d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:52.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e29819cf-d405-4eb2-9973-f7f37f7d20d3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:54.264Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2382 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:51.881Z InvocationID=2d74ae9f-f0df-4c8d-bac2-04104fc15ac4 LogTimestamp=2021-05-28T00:13:54.263988Z ResponseDesc=Created Exiting. 2021-05-28T00:13:54.264Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:54.370Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:54.370356Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:54.370Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:54.370356Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:54.371Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:13:54.472Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5eed737e-9bc2-4507-af7a-3727178033b4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:54.472009Z InvocationID=0f454ebb-dcf6-47a7-9cdb-7252312827ee LogTimestamp=2021-05-28T00:13:54.472093Z Entering 2021-05-28T00:13:54.472Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5eed737e-9bc2-4507-af7a-3727178033b4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:54.472009Z InvocationID=0f454ebb-dcf6-47a7-9cdb-7252312827ee LogTimestamp=2021-05-28T00:13:54.472093Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:54.666Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:54.472Z InvocationID=ed0065d5-a179-4317-89ce-652e607ee96f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:13:55.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=934b0559-7797-4983-a6c4-4d4d342f1afc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:55.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=934b0559-7797-4983-a6c4-4d4d342f1afc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16f5f183 2021-05-28T00:13:55.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=934b0559-7797-4983-a6c4-4d4d342f1afc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:55.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=934b0559-7797-4983-a6c4-4d4d342f1afc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:56.763Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2291 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:54.472Z InvocationID=ed0065d5-a179-4317-89ce-652e607ee96f LogTimestamp=2021-05-28T00:13:56.763401Z ResponseDesc=Created Exiting. 2021-05-28T00:13:56.764Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:56.868Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:56.868552Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:56.868Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:56.868552Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:56.869Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:13:56.971Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6f0074d3-a38b-4afc-b682-ec590ea3a7ac InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:56.970608Z InvocationID=dcce217c-bf68-4b75-818d-08df5455d575 LogTimestamp=2021-05-28T00:13:56.970703Z Entering 2021-05-28T00:13:56.971Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6f0074d3-a38b-4afc-b682-ec590ea3a7ac InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:56.970608Z InvocationID=dcce217c-bf68-4b75-818d-08df5455d575 LogTimestamp=2021-05-28T00:13:56.970703Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:57.160Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:56.971Z InvocationID=8ef39c0b-e149-4e9e-8ceb-6a6c9d561170 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:13:58.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd2c51e3-3152-4cde-ae07-1dde4cfcc8e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:13:58.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd2c51e3-3152-4cde-ae07-1dde4cfcc8e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5555a804 2021-05-28T00:13:58.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd2c51e3-3152-4cde-ae07-1dde4cfcc8e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:13:58.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd2c51e3-3152-4cde-ae07-1dde4cfcc8e4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:13:59.358Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2387 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:56.971Z InvocationID=8ef39c0b-e149-4e9e-8ceb-6a6c9d561170 LogTimestamp=2021-05-28T00:13:59.358446Z ResponseDesc=Created Exiting. 2021-05-28T00:13:59.359Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:13:59.471Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:59.471717Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:13:59.471Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:13:59.471717Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:13:59.472Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:13:59.574Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=49c9e70e-9fa4-4e0c-80a8-a394a5b387a8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:59.573983Z InvocationID=56adace7-0cc1-429e-a6c2-4bf9f06e5f95 LogTimestamp=2021-05-28T00:13:59.574060Z Entering 2021-05-28T00:13:59.574Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=49c9e70e-9fa4-4e0c-80a8-a394a5b387a8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:13:59.573983Z InvocationID=56adace7-0cc1-429e-a6c2-4bf9f06e5f95 LogTimestamp=2021-05-28T00:13:59.574060Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:13:59.669Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:59.574Z InvocationID=67e133c7-bbc3-4ff5-9451-ec44fb1d7b58 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:14:01.868Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2294 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:13:59.574Z InvocationID=67e133c7-bbc3-4ff5-9451-ec44fb1d7b58 LogTimestamp=2021-05-28T00:14:01.868161Z ResponseDesc=Created Exiting. 2021-05-28T00:14:01.868Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:14:01.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1421e5b1-97dd-4daa-83cf-a770c89d56bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:01.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1421e5b1-97dd-4daa-83cf-a770c89d56bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5774fcd8 2021-05-28T00:14:01.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1421e5b1-97dd-4daa-83cf-a770c89d56bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:01.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1421e5b1-97dd-4daa-83cf-a770c89d56bb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:02.071Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:14:02.071186Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:14:02.071Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:14:02.071186Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:14:02.071Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:14:02.275Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4027ff73-27a6-4efc-81b3-efab1d280c65 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:02.275067Z InvocationID=51af8063-445a-4fbf-8176-14b36f2713f7 LogTimestamp=2021-05-28T00:14:02.275129Z Entering 2021-05-28T00:14:02.275Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4027ff73-27a6-4efc-81b3-efab1d280c65 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:02.275067Z InvocationID=51af8063-445a-4fbf-8176-14b36f2713f7 LogTimestamp=2021-05-28T00:14:02.275129Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:14:02.369Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:14:02.275Z InvocationID=d3104c2d-03bf-4cdc-a5d8-613a3b266279 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:14:04.373Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2097 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:14:02.275Z InvocationID=d3104c2d-03bf-4cdc-a5d8-613a3b266279 LogTimestamp=2021-05-28T00:14:04.372734Z ResponseDesc=Created Exiting. 2021-05-28T00:14:04.373Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:14:04.860Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0f81cd53-a41d-434a-a1b8-89d1be618296 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:04.860315Z InvocationID=46da4703-efcd-402a-97e8-cc4b8206fa7a LogTimestamp=2021-05-28T00:14:04.860396Z Entering 2021-05-28T00:14:04.860Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0f81cd53-a41d-434a-a1b8-89d1be618296 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:04.860315Z InvocationID=46da4703-efcd-402a-97e8-cc4b8206fa7a LogTimestamp=2021-05-28T00:14:04.860396Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:14:04.877Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=16 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:14:04.861Z InvocationID=6f16262b-439b-49b3-a069-9a78f5a90fc2 LogTimestamp=2021-05-28T00:14:04.877251Z ResponseDesc=OK Exiting. 2021-05-28T00:14:04.878Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:14:04.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f8f9500-09bf-4a6a-8d9d-4f1a54d5685a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:04.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f8f9500-09bf-4a6a-8d9d-4f1a54d5685a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b616a66 2021-05-28T00:14:04.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f8f9500-09bf-4a6a-8d9d-4f1a54d5685a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:04.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f8f9500-09bf-4a6a-8d9d-4f1a54d5685a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:05.370Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=be0f9ce8-5185-44a1-8ffa-786c38bb9c9a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:05.370308Z InvocationID=b5ddf372-48c7-4ecf-a4b1-fa3ffa044edd LogTimestamp=2021-05-28T00:14:05.370375Z Entering 2021-05-28T00:14:05.370Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=be0f9ce8-5185-44a1-8ffa-786c38bb9c9a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:05.370308Z InvocationID=b5ddf372-48c7-4ecf-a4b1-fa3ffa044edd LogTimestamp=2021-05-28T00:14:05.370375Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:14:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:14:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:14:05.859Z [qtp580718781-12] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:14:05.370Z InvocationID=94499b3a-dd48-4fcd-bd71-51672de6bb1e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 486300057 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n486300057 100% \n 2021-05-28T00:14:05.869Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=499 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:14:05.370Z InvocationID=94499b3a-dd48-4fcd-bd71-51672de6bb1e LogTimestamp=2021-05-28T00:14:05.869086Z ResponseDesc=OK Exiting. 2021-05-28T00:14:05.869Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:14:05.965Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=934b67dc-a9bb-47bb-a4ab-161c53966903 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:05.964449Z InvocationID=54fcc853-1866-41d4-820f-9f5b3b566d11 LogTimestamp=2021-05-28T00:14:05.964538Z Entering 2021-05-28T00:14:05.965Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=934b67dc-a9bb-47bb-a4ab-161c53966903 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:05.964449Z InvocationID=54fcc853-1866-41d4-820f-9f5b3b566d11 LogTimestamp=2021-05-28T00:14:05.964538Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:14:06.286Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:14:05.965Z InvocationID=92308311-3026-4d46-9bf2-009f9f022bbe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:14:06.467Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:05.965Z InvocationID=92308311-3026-4d46-9bf2-009f9f022bbe ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:14:06.468Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:05.965Z InvocationID=92308311-3026-4d46-9bf2-009f9f022bbe ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:14:06.469Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=503 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:14:05.965Z InvocationID=92308311-3026-4d46-9bf2-009f9f022bbe ErrorCode=500 LogTimestamp=2021-05-28T00:14:06.468658Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:14:06.469Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:14:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a71c814e-5982-411b-854f-53c67763bf2e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a71c814e-5982-411b-854f-53c67763bf2e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5221f760 2021-05-28T00:14:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a71c814e-5982-411b-854f-53c67763bf2e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a71c814e-5982-411b-854f-53c67763bf2e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:10.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e98631-0f2a-4d71-96c2-0e9850d40904 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:10.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e98631-0f2a-4d71-96c2-0e9850d40904 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2acc60a8 2021-05-28T00:14:10.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e98631-0f2a-4d71-96c2-0e9850d40904 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:10.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e98631-0f2a-4d71-96c2-0e9850d40904 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:13.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b70d56e-311d-437f-9677-ccc0f55b6b06 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:13.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b70d56e-311d-437f-9677-ccc0f55b6b06 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a2edb15 2021-05-28T00:14:13.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b70d56e-311d-437f-9677-ccc0f55b6b06 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:13.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b70d56e-311d-437f-9677-ccc0f55b6b06 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:16.665Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9cd685ce-c894-40a3-bdcd-45b13d9d8a41 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:16.664832Z InvocationID=fe4a9819-fbca-4c23-aed4-ffb99203227f LogTimestamp=2021-05-28T00:14:16.664981Z Entering 2021-05-28T00:14:16.665Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9cd685ce-c894-40a3-bdcd-45b13d9d8a41 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:16.664832Z InvocationID=fe4a9819-fbca-4c23-aed4-ffb99203227f LogTimestamp=2021-05-28T00:14:16.664981Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:14:16.788Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:14:16.665Z InvocationID=74bfd0db-5518-4a97-99c4-0b5d20acd779 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:14:16.862Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:16.665Z InvocationID=74bfd0db-5518-4a97-99c4-0b5d20acd779 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:14:16.862Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:16.665Z InvocationID=74bfd0db-5518-4a97-99c4-0b5d20acd779 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:14:16.866Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=200 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:14:16.665Z InvocationID=74bfd0db-5518-4a97-99c4-0b5d20acd779 ErrorCode=500 LogTimestamp=2021-05-28T00:14:16.865663Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:14:16.866Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:14:16.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f51ed88-b045-499b-a0fe-c9278b3a920d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:16.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f51ed88-b045-499b-a0fe-c9278b3a920d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a7ca86c 2021-05-28T00:14:16.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f51ed88-b045-499b-a0fe-c9278b3a920d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:16.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f51ed88-b045-499b-a0fe-c9278b3a920d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:19.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be581aef-0be4-4f72-a6a3-c8ca59db5481 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:19.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be581aef-0be4-4f72-a6a3-c8ca59db5481 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67bbd2b4 2021-05-28T00:14:19.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be581aef-0be4-4f72-a6a3-c8ca59db5481 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:19.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be581aef-0be4-4f72-a6a3-c8ca59db5481 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:14:20.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:14:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4431a612-bedf-4a98-af0b-f8909972b754 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4431a612-bedf-4a98-af0b-f8909972b754 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ebb27cb 2021-05-28T00:14:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4431a612-bedf-4a98-af0b-f8909972b754 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4431a612-bedf-4a98-af0b-f8909972b754 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e10186c-f22f-49b0-9a59-83b60ec78930 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e10186c-f22f-49b0-9a59-83b60ec78930 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e1a77a 2021-05-28T00:14:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e10186c-f22f-49b0-9a59-83b60ec78930 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e10186c-f22f-49b0-9a59-83b60ec78930 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:26.978Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=67475365-c996-488b-b586-96b70b3ff154 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:26.977967Z InvocationID=2a70b3a1-1db4-4c4b-9132-689758e4d5e2 LogTimestamp=2021-05-28T00:14:26.978098Z Entering 2021-05-28T00:14:26.978Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=67475365-c996-488b-b586-96b70b3ff154 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:26.977967Z InvocationID=2a70b3a1-1db4-4c4b-9132-689758e4d5e2 LogTimestamp=2021-05-28T00:14:26.978098Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:14:27.086Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:14:26.978Z InvocationID=0c1afadf-45cc-43ad-bf45-5f526036769a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:14:27.172Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:26.978Z InvocationID=0c1afadf-45cc-43ad-bf45-5f526036769a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:14:27.172Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:26.978Z InvocationID=0c1afadf-45cc-43ad-bf45-5f526036769a ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:14:27.173Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=195 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:14:26.978Z InvocationID=0c1afadf-45cc-43ad-bf45-5f526036769a ErrorCode=500 LogTimestamp=2021-05-28T00:14:27.173083Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:14:27.173Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:14:28.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e405516f-ccfd-4d2e-87bf-2384de9ebcd1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:28.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e405516f-ccfd-4d2e-87bf-2384de9ebcd1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d814082 2021-05-28T00:14:28.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e405516f-ccfd-4d2e-87bf-2384de9ebcd1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:28.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e405516f-ccfd-4d2e-87bf-2384de9ebcd1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:31.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48762bf-18f5-467c-99ed-71365e92b502 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:31.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48762bf-18f5-467c-99ed-71365e92b502 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@649607f4 2021-05-28T00:14:31.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48762bf-18f5-467c-99ed-71365e92b502 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:31.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48762bf-18f5-467c-99ed-71365e92b502 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d65221cf-b11f-40ce-bd4c-0d851bf97723 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d65221cf-b11f-40ce-bd4c-0d851bf97723 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13e4e1d2 2021-05-28T00:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d65221cf-b11f-40ce-bd4c-0d851bf97723 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d65221cf-b11f-40ce-bd4c-0d851bf97723 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:14:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:14:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:14:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:14:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:14:37.275Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5f92c17b-029a-42dd-9766-8c0a2d49c22e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:37.275361Z InvocationID=847624ee-e322-425f-a05f-212c407c4a98 LogTimestamp=2021-05-28T00:14:37.275456Z Entering 2021-05-28T00:14:37.276Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5f92c17b-029a-42dd-9766-8c0a2d49c22e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:37.275361Z InvocationID=847624ee-e322-425f-a05f-212c407c4a98 LogTimestamp=2021-05-28T00:14:37.275456Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:14:37.380Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:14:37.276Z InvocationID=853a8876-9a30-4017-b112-24ba9e78085c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:14:37.560Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:37.276Z InvocationID=853a8876-9a30-4017-b112-24ba9e78085c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:14:37.560Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:37.276Z InvocationID=853a8876-9a30-4017-b112-24ba9e78085c ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:14:37.561Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=285 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:14:37.276Z InvocationID=853a8876-9a30-4017-b112-24ba9e78085c ErrorCode=500 LogTimestamp=2021-05-28T00:14:37.561625Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:14:37.562Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:14:37.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44847daa-8dfd-4d67-a2d9-190bf1284efa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:37.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44847daa-8dfd-4d67-a2d9-190bf1284efa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cb14ea3 2021-05-28T00:14:37.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44847daa-8dfd-4d67-a2d9-190bf1284efa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:37.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44847daa-8dfd-4d67-a2d9-190bf1284efa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:40.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad53a28f-8300-4d9f-b78c-9132cacd1d78 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:40.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad53a28f-8300-4d9f-b78c-9132cacd1d78 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4be388e7 2021-05-28T00:14:40.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad53a28f-8300-4d9f-b78c-9132cacd1d78 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:40.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad53a28f-8300-4d9f-b78c-9132cacd1d78 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:44.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcbb348d-7038-45b8-a8fc-47adafd4dad2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:44.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcbb348d-7038-45b8-a8fc-47adafd4dad2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f1e8fc8 2021-05-28T00:14:44.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcbb348d-7038-45b8-a8fc-47adafd4dad2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:44.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcbb348d-7038-45b8-a8fc-47adafd4dad2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:47.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172d8fb0-4507-44c3-88c6-1df9f2bfda6a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:47.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172d8fb0-4507-44c3-88c6-1df9f2bfda6a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bc01788 2021-05-28T00:14:47.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172d8fb0-4507-44c3-88c6-1df9f2bfda6a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:47.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=172d8fb0-4507-44c3-88c6-1df9f2bfda6a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:47.768Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c29f0ced-bd78-4f5e-9db8-080ac2b94bbb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:47.768017Z InvocationID=c6f18587-87bd-4754-a215-cd172b246279 LogTimestamp=2021-05-28T00:14:47.768171Z Entering 2021-05-28T00:14:47.768Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c29f0ced-bd78-4f5e-9db8-080ac2b94bbb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:47.768017Z InvocationID=c6f18587-87bd-4754-a215-cd172b246279 LogTimestamp=2021-05-28T00:14:47.768171Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:14:47.877Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:14:47.769Z InvocationID=c69ed1df-a76c-44a7-8ce5-65c95a853914 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:14:47.963Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:47.769Z InvocationID=c69ed1df-a76c-44a7-8ce5-65c95a853914 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:14:47.963Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:47.769Z InvocationID=c69ed1df-a76c-44a7-8ce5-65c95a853914 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:14:47.964Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=195 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:14:47.769Z InvocationID=c69ed1df-a76c-44a7-8ce5-65c95a853914 ErrorCode=500 LogTimestamp=2021-05-28T00:14:47.964598Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:14:47.965Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:14:49.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3ca634c-68aa-41b5-b480-b0c68f883fec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:49.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3ca634c-68aa-41b5-b480-b0c68f883fec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d128005 2021-05-28T00:14:49.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3ca634c-68aa-41b5-b480-b0c68f883fec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:49.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3ca634c-68aa-41b5-b480-b0c68f883fec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:14:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:14:52.224Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:14:52.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59613a34-2022-4c97-beba-579af6c555e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:52.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59613a34-2022-4c97-beba-579af6c555e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ae193f0 2021-05-28T00:14:52.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59613a34-2022-4c97-beba-579af6c555e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:52.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59613a34-2022-4c97-beba-579af6c555e9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:55.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4609de4e-1ec5-4194-bc4c-1b1adafb8afa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:55.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4609de4e-1ec5-4194-bc4c-1b1adafb8afa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16a31ab 2021-05-28T00:14:55.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4609de4e-1ec5-4194-bc4c-1b1adafb8afa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:55.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4609de4e-1ec5-4194-bc4c-1b1adafb8afa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:14:58.367Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6c89f99c-3d8c-4239-b1a3-04bfa8ad99df InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:58.366639Z InvocationID=5af38404-3baf-4285-8870-dd09a3a3d8b2 LogTimestamp=2021-05-28T00:14:58.366758Z Entering 2021-05-28T00:14:58.367Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6c89f99c-3d8c-4239-b1a3-04bfa8ad99df InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:14:58.366639Z InvocationID=5af38404-3baf-4285-8870-dd09a3a3d8b2 LogTimestamp=2021-05-28T00:14:58.366758Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:14:58.472Z [qtp580718781-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:14:58.367Z InvocationID=9ddabe85-31e6-4993-a245-c5669f50f1e3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:14:58.658Z [qtp580718781-17] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:58.367Z InvocationID=9ddabe85-31e6-4993-a245-c5669f50f1e3 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:14:58.658Z [qtp580718781-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:14:58.367Z InvocationID=9ddabe85-31e6-4993-a245-c5669f50f1e3 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:14:58.659Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=292 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:14:58.367Z InvocationID=9ddabe85-31e6-4993-a245-c5669f50f1e3 ErrorCode=500 LogTimestamp=2021-05-28T00:14:58.659417Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:14:58.660Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:14:59.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a9fb2fb-395c-4791-b84b-f43d4f09e025 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:14:59.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a9fb2fb-395c-4791-b84b-f43d4f09e025 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a3d0a59 2021-05-28T00:14:59.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a9fb2fb-395c-4791-b84b-f43d4f09e025 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:14:59.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a9fb2fb-395c-4791-b84b-f43d4f09e025 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:01.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c976e3eb-a4b3-423a-8fa6-c9c16f194f97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:01.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c976e3eb-a4b3-423a-8fa6-c9c16f194f97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@386a5b3 2021-05-28T00:15:01.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c976e3eb-a4b3-423a-8fa6-c9c16f194f97 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:01.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c976e3eb-a4b3-423a-8fa6-c9c16f194f97 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:04.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1add40f4-876d-46b5-b525-5ac4bdda8e22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:04.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1add40f4-876d-46b5-b525-5ac4bdda8e22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@659ec820 2021-05-28T00:15:04.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1add40f4-876d-46b5-b525-5ac4bdda8e22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:04.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1add40f4-876d-46b5-b525-5ac4bdda8e22 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:15:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:15:07.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995acbff-d501-4b6b-9522-752e244b05f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:07.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995acbff-d501-4b6b-9522-752e244b05f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35a94a51 2021-05-28T00:15:07.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995acbff-d501-4b6b-9522-752e244b05f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:07.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995acbff-d501-4b6b-9522-752e244b05f4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:08.886Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=28ac5754-0c66-44d5-b037-5c06be6b7dc2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:08.886Z InvocationID=7560b84d-b0aa-4273-a8dc-2f52a44cb5f5 LogTimestamp=2021-05-28T00:15:08.886112Z Entering 2021-05-28T00:15:08.886Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=28ac5754-0c66-44d5-b037-5c06be6b7dc2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:08.886Z InvocationID=7560b84d-b0aa-4273-a8dc-2f52a44cb5f5 LogTimestamp=2021-05-28T00:15:08.886112Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:15:08.984Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:15:08.886Z InvocationID=025550de-8844-4367-b957-0ba133e9df7f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:15:09.175Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:08.886Z InvocationID=025550de-8844-4367-b957-0ba133e9df7f ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:15:09.175Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:08.886Z InvocationID=025550de-8844-4367-b957-0ba133e9df7f ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:15:09.176Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=290 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:15:08.886Z InvocationID=025550de-8844-4367-b957-0ba133e9df7f ErrorCode=500 LogTimestamp=2021-05-28T00:15:09.176199Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:15:09.176Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:15:10.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d3858c5-5510-452e-ae92-aea159bbb9d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d3858c5-5510-452e-ae92-aea159bbb9d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7784f17f 2021-05-28T00:15:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d3858c5-5510-452e-ae92-aea159bbb9d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d3858c5-5510-452e-ae92-aea159bbb9d9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2eba5a06-c795-443a-b4ff-964987cb0ec0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2eba5a06-c795-443a-b4ff-964987cb0ec0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41197901 2021-05-28T00:15:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2eba5a06-c795-443a-b4ff-964987cb0ec0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2eba5a06-c795-443a-b4ff-964987cb0ec0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:17.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e45ab8b9-b902-4d45-8af8-56b3e140eaa5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:17.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e45ab8b9-b902-4d45-8af8-56b3e140eaa5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@525576ca 2021-05-28T00:15:17.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e45ab8b9-b902-4d45-8af8-56b3e140eaa5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:17.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e45ab8b9-b902-4d45-8af8-56b3e140eaa5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:19.281Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0d114516-c56a-49ae-8155-870843b238dc InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:19.281301Z InvocationID=3a5e5c46-3b3f-41d8-860f-2d8cf894c9a4 LogTimestamp=2021-05-28T00:15:19.281505Z Entering 2021-05-28T00:15:19.282Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0d114516-c56a-49ae-8155-870843b238dc InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:19.281301Z InvocationID=3a5e5c46-3b3f-41d8-860f-2d8cf894c9a4 LogTimestamp=2021-05-28T00:15:19.281505Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:15:19.455Z [qtp580718781-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:15:19.282Z InvocationID=7c6db9eb-84be-4a3f-8161-048dc02cee47 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:15:19.477Z [qtp580718781-17] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:19.282Z InvocationID=7c6db9eb-84be-4a3f-8161-048dc02cee47 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:15:19.477Z [qtp580718781-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:19.282Z InvocationID=7c6db9eb-84be-4a3f-8161-048dc02cee47 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:15:19.478Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=196 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:15:19.282Z InvocationID=7c6db9eb-84be-4a3f-8161-048dc02cee47 ErrorCode=500 LogTimestamp=2021-05-28T00:15:19.478258Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:15:19.479Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:15:19.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e02cca7-5ab1-48f7-9230-2329c306c499 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:19.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e02cca7-5ab1-48f7-9230-2329c306c499 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d514513 2021-05-28T00:15:19.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e02cca7-5ab1-48f7-9230-2329c306c499 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:19.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e02cca7-5ab1-48f7-9230-2329c306c499 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:15:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:15:22.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f63fc5-fc7e-4ee9-892a-698234b5e8a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:22.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f63fc5-fc7e-4ee9-892a-698234b5e8a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d8e56cf 2021-05-28T00:15:22.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f63fc5-fc7e-4ee9-892a-698234b5e8a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:22.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f63fc5-fc7e-4ee9-892a-698234b5e8a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac7b53dd-a283-4d90-8da3-442b78f531b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac7b53dd-a283-4d90-8da3-442b78f531b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7755f90e 2021-05-28T00:15:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac7b53dd-a283-4d90-8da3-442b78f531b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac7b53dd-a283-4d90-8da3-442b78f531b4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:28.998Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12924394-d42b-4cbc-a084-3f22074013b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:28.998Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12924394-d42b-4cbc-a084-3f22074013b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e4f3208 2021-05-28T00:15:28.998Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12924394-d42b-4cbc-a084-3f22074013b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:28.998Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12924394-d42b-4cbc-a084-3f22074013b3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:29.685Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a95ed27a-16a2-4d79-8f7a-b8e6507b7723 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:29.685030Z InvocationID=c30edb77-1ec9-442a-9867-a366d13f55ee LogTimestamp=2021-05-28T00:15:29.685095Z Entering 2021-05-28T00:15:29.685Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a95ed27a-16a2-4d79-8f7a-b8e6507b7723 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:29.685030Z InvocationID=c30edb77-1ec9-442a-9867-a366d13f55ee LogTimestamp=2021-05-28T00:15:29.685095Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:15:29.795Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:15:29.685Z InvocationID=e4f8d811-9fa1-4d1d-ba14-ce6b63c86e96 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:15:29.881Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:29.685Z InvocationID=e4f8d811-9fa1-4d1d-ba14-ce6b63c86e96 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:15:29.882Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:29.685Z InvocationID=e4f8d811-9fa1-4d1d-ba14-ce6b63c86e96 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:15:29.883Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=198 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:15:29.685Z InvocationID=e4f8d811-9fa1-4d1d-ba14-ce6b63c86e96 ErrorCode=500 LogTimestamp=2021-05-28T00:15:29.883490Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:15:29.884Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:15:32.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=516f3220-b2fa-4440-941d-2fed1f8d4f3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:32.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=516f3220-b2fa-4440-941d-2fed1f8d4f3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2911870f 2021-05-28T00:15:32.191Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=516f3220-b2fa-4440-941d-2fed1f8d4f3b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:32.191Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=516f3220-b2fa-4440-941d-2fed1f8d4f3b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:34.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=657a3b7a-6e68-4430-ae8a-2ea743e3b7d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:34.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=657a3b7a-6e68-4430-ae8a-2ea743e3b7d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@262684cc 2021-05-28T00:15:34.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=657a3b7a-6e68-4430-ae8a-2ea743e3b7d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:34.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=657a3b7a-6e68-4430-ae8a-2ea743e3b7d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:15:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:15:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:15:37.165Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:15:37.165Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:15:37.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec111978-e306-45db-97da-09f6df759386 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:37.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec111978-e306-45db-97da-09f6df759386 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e63848f 2021-05-28T00:15:37.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec111978-e306-45db-97da-09f6df759386 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:37.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec111978-e306-45db-97da-09f6df759386 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:39.981Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=94e11c03-2b2b-4de9-8b4e-04d6a5e5f46c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:39.981587Z InvocationID=a161f261-9f2a-4538-bba7-1797b490a6b1 LogTimestamp=2021-05-28T00:15:39.981705Z Entering 2021-05-28T00:15:39.982Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=94e11c03-2b2b-4de9-8b4e-04d6a5e5f46c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:39.981587Z InvocationID=a161f261-9f2a-4538-bba7-1797b490a6b1 LogTimestamp=2021-05-28T00:15:39.981705Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:15:40.126Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:15:39.982Z InvocationID=0ab29079-dad2-4999-bcf7-f27a4520489e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:15:40.358Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:39.982Z InvocationID=0ab29079-dad2-4999-bcf7-f27a4520489e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:15:40.358Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:39.982Z InvocationID=0ab29079-dad2-4999-bcf7-f27a4520489e ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:15:40.360Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=377 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:15:39.982Z InvocationID=0ab29079-dad2-4999-bcf7-f27a4520489e ErrorCode=500 LogTimestamp=2021-05-28T00:15:40.359736Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:15:40.360Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:15:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc91edc-3a13-4fdc-8d66-7e5b86073569 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc91edc-3a13-4fdc-8d66-7e5b86073569 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@94b2639 2021-05-28T00:15:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc91edc-3a13-4fdc-8d66-7e5b86073569 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:40.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bc91edc-3a13-4fdc-8d66-7e5b86073569 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:43.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9c5c256-293d-4640-bc5e-43b809f29171 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:43.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9c5c256-293d-4640-bc5e-43b809f29171 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c47e1e0 2021-05-28T00:15:43.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9c5c256-293d-4640-bc5e-43b809f29171 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:43.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9c5c256-293d-4640-bc5e-43b809f29171 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f88050d-c511-4fef-be9a-a71781a84aae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f88050d-c511-4fef-be9a-a71781a84aae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ed51040 2021-05-28T00:15:46.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f88050d-c511-4fef-be9a-a71781a84aae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:46.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f88050d-c511-4fef-be9a-a71781a84aae PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:49.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a8b0ed-961e-40ae-96b3-11f870f3cbf7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:49.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a8b0ed-961e-40ae-96b3-11f870f3cbf7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cf0ace9 2021-05-28T00:15:49.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a8b0ed-961e-40ae-96b3-11f870f3cbf7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:49.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a8b0ed-961e-40ae-96b3-11f870f3cbf7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:50.467Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=24219f67-11c7-4ca0-a6dd-3d0459476023 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:50.466576Z InvocationID=7a35eb69-ebed-4ba7-b0c5-1f8c513ddf16 LogTimestamp=2021-05-28T00:15:50.466679Z Entering 2021-05-28T00:15:50.467Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=24219f67-11c7-4ca0-a6dd-3d0459476023 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.175 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.175 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:15:50.466576Z InvocationID=7a35eb69-ebed-4ba7-b0c5-1f8c513ddf16 LogTimestamp=2021-05-28T00:15:50.466679Z localAddr=10.242.102.221 remoteAddr=10.242.170.175 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:15:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:15:50.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:15:50.581Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:15:50.467Z InvocationID=9486b388-4b91-49d7-a380-7b3170dbf357 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:15:50.696Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:50.467Z InvocationID=9486b388-4b91-49d7-a380-7b3170dbf357 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:15:50.696Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:15:50.467Z InvocationID=9486b388-4b91-49d7-a380-7b3170dbf357 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:15:50.697Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=230 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:15:50.467Z InvocationID=9486b388-4b91-49d7-a380-7b3170dbf357 ErrorCode=500 LogTimestamp=2021-05-28T00:15:50.697390Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:15:50.697Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:15:52.123Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:15:52.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6729f3-265b-4659-87c2-f1b4ed080f9a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:52.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6729f3-265b-4659-87c2-f1b4ed080f9a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c5a33a0 2021-05-28T00:15:52.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6729f3-265b-4659-87c2-f1b4ed080f9a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:52.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f6729f3-265b-4659-87c2-f1b4ed080f9a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:55.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6ce8f6-ab46-4dd4-a55d-7b61f7634319 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:15:55.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6ce8f6-ab46-4dd4-a55d-7b61f7634319 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f818b2a 2021-05-28T00:15:55.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6ce8f6-ab46-4dd4-a55d-7b61f7634319 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:15:55.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6ce8f6-ab46-4dd4-a55d-7b61f7634319 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:15:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:15:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:16:01.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e35cdef8-5a16-444f-874c-d22867e0b5e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:01.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e35cdef8-5a16-444f-874c-d22867e0b5e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@940cf8a 2021-05-28T00:16:01.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e35cdef8-5a16-444f-874c-d22867e0b5e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:01.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e35cdef8-5a16-444f-874c-d22867e0b5e9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:01.771Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:16:02.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5cb24ac-792c-4b77-bdd8-0e2150980d55 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:02.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5cb24ac-792c-4b77-bdd8-0e2150980d55 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@500eadaf 2021-05-28T00:16:02.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5cb24ac-792c-4b77-bdd8-0e2150980d55 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:02.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5cb24ac-792c-4b77-bdd8-0e2150980d55 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:04.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b944ab0-ca41-4357-98e9-5c2ffda5f52a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:04.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b944ab0-ca41-4357-98e9-5c2ffda5f52a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d94533d 2021-05-28T00:16:04.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b944ab0-ca41-4357-98e9-5c2ffda5f52a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:04.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b944ab0-ca41-4357-98e9-5c2ffda5f52a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:16:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:16:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ca8f10-a4c0-40a1-a019-b7870b03a1a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ca8f10-a4c0-40a1-a019-b7870b03a1a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e5f517e 2021-05-28T00:16:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ca8f10-a4c0-40a1-a019-b7870b03a1a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ca8f10-a4c0-40a1-a019-b7870b03a1a8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d630d291-1129-4af4-beab-d4ec9e00c20e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d630d291-1129-4af4-beab-d4ec9e00c20e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11ecf1a0 2021-05-28T00:16:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d630d291-1129-4af4-beab-d4ec9e00c20e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d630d291-1129-4af4-beab-d4ec9e00c20e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:14.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a6d230b-ef18-41eb-b96b-a00aa9a65b8e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:14.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a6d230b-ef18-41eb-b96b-a00aa9a65b8e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34aa55e7 2021-05-28T00:16:14.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a6d230b-ef18-41eb-b96b-a00aa9a65b8e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:14.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a6d230b-ef18-41eb-b96b-a00aa9a65b8e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:16.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9de9a2de-23b5-489d-9358-742f8d114150 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:16.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9de9a2de-23b5-489d-9358-742f8d114150 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b08f5f4 2021-05-28T00:16:16.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9de9a2de-23b5-489d-9358-742f8d114150 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:16.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9de9a2de-23b5-489d-9358-742f8d114150 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:19.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b9227a4-db37-41d8-adc1-38bc721d9755 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:19.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b9227a4-db37-41d8-adc1-38bc721d9755 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f38834a 2021-05-28T00:16:19.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b9227a4-db37-41d8-adc1-38bc721d9755 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:19.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b9227a4-db37-41d8-adc1-38bc721d9755 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:16:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:16:22.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5315d7cd-d19a-4f48-98c8-20c5f1035914 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:22.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5315d7cd-d19a-4f48-98c8-20c5f1035914 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9af0cfc 2021-05-28T00:16:22.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5315d7cd-d19a-4f48-98c8-20c5f1035914 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:22.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5315d7cd-d19a-4f48-98c8-20c5f1035914 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:25.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=087234b4-3181-43d8-a444-1d601875cf3a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:25.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=087234b4-3181-43d8-a444-1d601875cf3a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f4c6543 2021-05-28T00:16:25.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=087234b4-3181-43d8-a444-1d601875cf3a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:25.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=087234b4-3181-43d8-a444-1d601875cf3a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:28.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e30763-5b20-480c-8d31-188c6eb7bbaf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:28.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e30763-5b20-480c-8d31-188c6eb7bbaf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33b78a0b 2021-05-28T00:16:28.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e30763-5b20-480c-8d31-188c6eb7bbaf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:28.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e30763-5b20-480c-8d31-188c6eb7bbaf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:32.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66823649-74c6-4b59-90ac-3096118a334a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:32.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66823649-74c6-4b59-90ac-3096118a334a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8ae6de4 2021-05-28T00:16:32.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66823649-74c6-4b59-90ac-3096118a334a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:32.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66823649-74c6-4b59-90ac-3096118a334a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:35.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:35.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@791758e1 2021-05-28T00:16:35.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:35.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:35.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:16:35.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:16:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:16:37.165Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:16:37.165Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:16:38.280Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:16:38.281Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:16:38.281Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:16:38.281Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a16dd2f4-f64c-44a1-9b72-acedb088ed23 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:16:40.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ecd3a1f-6d7f-4ef9-9556-3b836bb6c7be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:40.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ecd3a1f-6d7f-4ef9-9556-3b836bb6c7be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75fd5cfd 2021-05-28T00:16:40.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ecd3a1f-6d7f-4ef9-9556-3b836bb6c7be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:40.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ecd3a1f-6d7f-4ef9-9556-3b836bb6c7be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:41.074Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6ecd3a1f-6d7f-4ef9-9556-3b836bb6c7be ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:16:41.074Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6ecd3a1f-6d7f-4ef9-9556-3b836bb6c7be ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:16:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab725f32-523d-4fe9-b902-6808bf48a37d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab725f32-523d-4fe9-b902-6808bf48a37d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12fe470 2021-05-28T00:16:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab725f32-523d-4fe9-b902-6808bf48a37d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab725f32-523d-4fe9-b902-6808bf48a37d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:45.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f6e8d37-712c-4bd7-9d91-11a63b1e19d2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:45.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f6e8d37-712c-4bd7-9d91-11a63b1e19d2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42bc74e4 2021-05-28T00:16:45.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f6e8d37-712c-4bd7-9d91-11a63b1e19d2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:45.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f6e8d37-712c-4bd7-9d91-11a63b1e19d2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:47.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bc2de4d-7d6e-4f70-ad00-fbf843d88c71 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:47.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bc2de4d-7d6e-4f70-ad00-fbf843d88c71 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d39c2fa 2021-05-28T00:16:47.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bc2de4d-7d6e-4f70-ad00-fbf843d88c71 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:47.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bc2de4d-7d6e-4f70-ad00-fbf843d88c71 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:49.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbd45f8-5028-4922-a30e-6f8a0bb4c7c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:49.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbd45f8-5028-4922-a30e-6f8a0bb4c7c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cc15f79 2021-05-28T00:16:49.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbd45f8-5028-4922-a30e-6f8a0bb4c7c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:49.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbd45f8-5028-4922-a30e-6f8a0bb4c7c7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:16:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:16:52.134Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:16:52.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ba636b0-cb54-45b2-bdcb-830d1ac35b3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:52.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ba636b0-cb54-45b2-bdcb-830d1ac35b3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@723326f5 2021-05-28T00:16:52.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ba636b0-cb54-45b2-bdcb-830d1ac35b3b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:52.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ba636b0-cb54-45b2-bdcb-830d1ac35b3b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:55.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cdc66b7-304c-489f-997f-e3ccf48404de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:55.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cdc66b7-304c-489f-997f-e3ccf48404de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53096703 2021-05-28T00:16:55.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cdc66b7-304c-489f-997f-e3ccf48404de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:55.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cdc66b7-304c-489f-997f-e3ccf48404de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:16:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e32129f2-290f-4760-a437-caabc537486d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:16:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e32129f2-290f-4760-a437-caabc537486d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40b4518a 2021-05-28T00:16:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e32129f2-290f-4760-a437-caabc537486d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:16:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e32129f2-290f-4760-a437-caabc537486d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:02.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817d3ce2-72ac-4589-8fa0-2a3f7be71259 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:02.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817d3ce2-72ac-4589-8fa0-2a3f7be71259 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4608e472 2021-05-28T00:17:02.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817d3ce2-72ac-4589-8fa0-2a3f7be71259 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:02.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817d3ce2-72ac-4589-8fa0-2a3f7be71259 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:04.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e4e3a64-34c5-4785-9d87-defd0cbf5ad7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:04.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e4e3a64-34c5-4785-9d87-defd0cbf5ad7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2de766c8 2021-05-28T00:17:04.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e4e3a64-34c5-4785-9d87-defd0cbf5ad7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:04.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e4e3a64-34c5-4785-9d87-defd0cbf5ad7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:17:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:17:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b10ad7a-c374-48ac-8d74-8c0938edfb83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b10ad7a-c374-48ac-8d74-8c0938edfb83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35ce3d08 2021-05-28T00:17:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b10ad7a-c374-48ac-8d74-8c0938edfb83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:07.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b10ad7a-c374-48ac-8d74-8c0938edfb83 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:10.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb4bee1a-56ab-41ce-aae3-33a11ca1f14d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb4bee1a-56ab-41ce-aae3-33a11ca1f14d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5794123a 2021-05-28T00:17:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb4bee1a-56ab-41ce-aae3-33a11ca1f14d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb4bee1a-56ab-41ce-aae3-33a11ca1f14d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:14.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28296a1d-4d23-4dcd-94fa-bec317129f5a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:14.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28296a1d-4d23-4dcd-94fa-bec317129f5a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56be57ce 2021-05-28T00:17:14.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28296a1d-4d23-4dcd-94fa-bec317129f5a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:14.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28296a1d-4d23-4dcd-94fa-bec317129f5a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:16.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e260ca-fcd7-4e26-a1c5-d5ca73351cb0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:16.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e260ca-fcd7-4e26-a1c5-d5ca73351cb0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40f9baf7 2021-05-28T00:17:16.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e260ca-fcd7-4e26-a1c5-d5ca73351cb0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:16.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e260ca-fcd7-4e26-a1c5-d5ca73351cb0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c88ad2ba-0eff-44ee-b1d3-d0a3697708ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c88ad2ba-0eff-44ee-b1d3-d0a3697708ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41be99d 2021-05-28T00:17:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c88ad2ba-0eff-44ee-b1d3-d0a3697708ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c88ad2ba-0eff-44ee-b1d3-d0a3697708ca PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:20.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:17:20.520Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:17:23.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18f9253-e9e4-4fb2-8d99-10810d0c0ec9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:23.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18f9253-e9e4-4fb2-8d99-10810d0c0ec9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73693ee6 2021-05-28T00:17:23.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18f9253-e9e4-4fb2-8d99-10810d0c0ec9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:23.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f18f9253-e9e4-4fb2-8d99-10810d0c0ec9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:26.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b40928-6905-4c5f-80c6-89d91430aa7d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:26.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b40928-6905-4c5f-80c6-89d91430aa7d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@574d0762 2021-05-28T00:17:26.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b40928-6905-4c5f-80c6-89d91430aa7d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:26.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b40928-6905-4c5f-80c6-89d91430aa7d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:29.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28faecd7-71b3-4f85-a89b-514beb120b99 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:29.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28faecd7-71b3-4f85-a89b-514beb120b99 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5aee317e 2021-05-28T00:17:29.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28faecd7-71b3-4f85-a89b-514beb120b99 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:29.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28faecd7-71b3-4f85-a89b-514beb120b99 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:31.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3422ae42-6a47-4d93-8db2-2e506c23a405 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:31.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3422ae42-6a47-4d93-8db2-2e506c23a405 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e26cbf 2021-05-28T00:17:31.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3422ae42-6a47-4d93-8db2-2e506c23a405 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:31.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3422ae42-6a47-4d93-8db2-2e506c23a405 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:17:35.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:17:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:17:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:17:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:17:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:17:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:17:37.979Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:17:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=034ec3be-eb88-4690-8d87-9d54efa1d2cb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=034ec3be-eb88-4690-8d87-9d54efa1d2cb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bc6fc6b 2021-05-28T00:17:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=034ec3be-eb88-4690-8d87-9d54efa1d2cb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=034ec3be-eb88-4690-8d87-9d54efa1d2cb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:40.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f13ee85-4564-4a13-a474-74a5384a329d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:40.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f13ee85-4564-4a13-a474-74a5384a329d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40f8a66 2021-05-28T00:17:40.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f13ee85-4564-4a13-a474-74a5384a329d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:40.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f13ee85-4564-4a13-a474-74a5384a329d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:43.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cedb39a-3755-4f79-918c-9e5706b7f6a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:43.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cedb39a-3755-4f79-918c-9e5706b7f6a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f412e58 2021-05-28T00:17:43.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cedb39a-3755-4f79-918c-9e5706b7f6a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:43.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cedb39a-3755-4f79-918c-9e5706b7f6a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:47.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=096dccb6-bedc-4f81-8d1b-f3744df89716 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:47.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=096dccb6-bedc-4f81-8d1b-f3744df89716 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dd4389b 2021-05-28T00:17:47.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=096dccb6-bedc-4f81-8d1b-f3744df89716 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:47.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=096dccb6-bedc-4f81-8d1b-f3744df89716 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:49.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf237bec-8105-4210-8846-4932ee45b3ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:49.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf237bec-8105-4210-8846-4932ee45b3ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@317dbeee 2021-05-28T00:17:49.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf237bec-8105-4210-8846-4932ee45b3ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:49.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf237bec-8105-4210-8846-4932ee45b3ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:17:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:17:50.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14b994f-cfa4-4460-84c9-23cf0787aa8c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:50.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14b994f-cfa4-4460-84c9-23cf0787aa8c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49594e1d 2021-05-28T00:17:50.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14b994f-cfa4-4460-84c9-23cf0787aa8c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:50.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14b994f-cfa4-4460-84c9-23cf0787aa8c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:51.689Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:17:52.574Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a8fd7ea5-3a45-4f05-a558-30cf7b0c60ae InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:52.573457Z InvocationID=92882044-eeea-4d3f-aeb1-04f73196747c LogTimestamp=2021-05-28T00:17:52.573687Z Entering 2021-05-28T00:17:52.574Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a8fd7ea5-3a45-4f05-a558-30cf7b0c60ae InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:52.573457Z InvocationID=92882044-eeea-4d3f-aeb1-04f73196747c LogTimestamp=2021-05-28T00:17:52.573687Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:17:52.858Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=283 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:17:52.575Z InvocationID=cab21ee8-74f1-4444-aabb-8a37ba27dc6d LogTimestamp=2021-05-28T00:17:52.858072Z ResponseDesc=OK Exiting. 2021-05-28T00:17:52.858Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:17:52.971Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=38139 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:17:14.831Z InvocationID=91d3b0fc-8eba-4950-be8c-d63cfc5f62d4 ErrorCode=500 LogTimestamp=2021-05-28T00:17:52.970995Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:52.972Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:17:53.071Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c0a9e6dc-5c46-4a11-9c80-be1d496cf0d3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:53.071189Z InvocationID=2ab43f64-f1c4-4799-a1bb-24d1926ebd48 LogTimestamp=2021-05-28T00:17:53.071247Z Entering 2021-05-28T00:17:53.071Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c0a9e6dc-5c46-4a11-9c80-be1d496cf0d3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:53.071189Z InvocationID=2ab43f64-f1c4-4799-a1bb-24d1926ebd48 LogTimestamp=2021-05-28T00:17:53.071247Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:17:53.480Z [qtp580718781-96] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:17:53.071Z InvocationID=bb4ab9f6-de49-45d1-802a-d61a2ec2985c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c198498', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 406ms 2021-05-28T00:17:53.481Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=410 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:17:53.071Z InvocationID=bb4ab9f6-de49-45d1-802a-d61a2ec2985c ErrorCode=409 LogTimestamp=2021-05-28T00:17:53.481385Z ResponseDesc=Conflict Exiting. 2021-05-28T00:17:53.482Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:17:53.580Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=46945 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:17:06.634Z InvocationID=2a363540-6bdc-4f3c-9462-829f0bbefd59 ErrorCode=500 LogTimestamp=2021-05-28T00:17:53.579987Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:53.580Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:17:53.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682c7d1e-8ab5-4263-8c22-b5474021267b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:53.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682c7d1e-8ab5-4263-8c22-b5474021267b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f4c8a80 2021-05-28T00:17:53.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682c7d1e-8ab5-4263-8c22-b5474021267b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:53.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682c7d1e-8ab5-4263-8c22-b5474021267b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:53.763Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6c78ed3d-5b62-4826-9982-e463436cb8aa InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:53.762879Z InvocationID=b3ebe60d-bd41-4230-b23e-8d58ab4696ee LogTimestamp=2021-05-28T00:17:53.762951Z Entering 2021-05-28T00:17:53.763Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6c78ed3d-5b62-4826-9982-e463436cb8aa InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:53.762879Z InvocationID=b3ebe60d-bd41-4230-b23e-8d58ab4696ee LogTimestamp=2021-05-28T00:17:53.762951Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:17:53.971Z [qtp580718781-18] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:17:53.763Z InvocationID=e876f2fd-2e6a-4408-8995-e8e4731450f3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c198498', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 206ms 2021-05-28T00:17:53.972Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=208 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:17:53.763Z InvocationID=e876f2fd-2e6a-4408-8995-e8e4731450f3 ErrorCode=400 LogTimestamp=2021-05-28T00:17:53.971988Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:17:53.972Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:17:54.072Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:54.072431Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:17:54.072Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:54.072431Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:54.073Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:17:54.174Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=42a16ea7-affa-4f03-a9d6-ae64f6c2eeb3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:54.173959Z InvocationID=683c0f8e-fd89-4fef-aeae-5f1dc364de94 LogTimestamp=2021-05-28T00:17:54.174029Z Entering 2021-05-28T00:17:54.174Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=42a16ea7-affa-4f03-a9d6-ae64f6c2eeb3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:54.173959Z InvocationID=683c0f8e-fd89-4fef-aeae-5f1dc364de94 LogTimestamp=2021-05-28T00:17:54.174029Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:17:54.282Z [qtp580718781-16] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:17:54.174Z InvocationID=c2031dd2-5c7f-47a0-851c-9dafe4520b3b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c198498', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 106ms 2021-05-28T00:17:54.283Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=109 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:17:54.174Z InvocationID=c2031dd2-5c7f-47a0-851c-9dafe4520b3b ErrorCode=400 LogTimestamp=2021-05-28T00:17:54.283113Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:17:54.283Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:17:54.378Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:54.378750Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:17:54.378Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:54.378750Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:54.379Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:17:54.570Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=09ee7f20-b8c1-4b20-be8a-0427e5ac90f6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:54.569993Z InvocationID=fed8b6c1-c800-4c1e-bc96-99e941fcf149 LogTimestamp=2021-05-28T00:17:54.570067Z Entering 2021-05-28T00:17:54.570Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=09ee7f20-b8c1-4b20-be8a-0427e5ac90f6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:54.569993Z InvocationID=fed8b6c1-c800-4c1e-bc96-99e941fcf149 LogTimestamp=2021-05-28T00:17:54.570067Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:17:54.758Z [qtp580718781-12] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:17:54.570Z InvocationID=15d145c6-c347-4d26-a9b2-dea674151b5f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c198498', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 181ms 2021-05-28T00:17:54.759Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=189 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:17:54.570Z InvocationID=15d145c6-c347-4d26-a9b2-dea674151b5f ErrorCode=409 LogTimestamp=2021-05-28T00:17:54.759259Z ResponseDesc=Conflict Exiting. 2021-05-28T00:17:54.759Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:17:54.969Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:54.969140Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:17:54.969Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:54.969140Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:54.969Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:17:55.068Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d3b1cf2e-e9ea-48f9-a82c-e78487e0402c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:55.068036Z InvocationID=8af1f4ea-7460-4361-920a-9815267f3aa1 LogTimestamp=2021-05-28T00:17:55.068114Z Entering 2021-05-28T00:17:55.068Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d3b1cf2e-e9ea-48f9-a82c-e78487e0402c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:55.068036Z InvocationID=8af1f4ea-7460-4361-920a-9815267f3aa1 LogTimestamp=2021-05-28T00:17:55.068114Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:17:55.660Z [qtp580718781-98] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:17:55.068Z InvocationID=515c3935-3ba2-4732-ba20-59c415569380 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c198498', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 590ms 2021-05-28T00:17:55.661Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=593 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:17:55.068Z InvocationID=515c3935-3ba2-4732-ba20-59c415569380 ErrorCode=409 LogTimestamp=2021-05-28T00:17:55.661087Z ResponseDesc=Conflict Exiting. 2021-05-28T00:17:55.661Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:17:55.877Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:55.876901Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:17:55.877Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:55.876901Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:55.877Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:17:56.071Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=595b4720-cce9-4fa3-bb4b-5117245fc953 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:56.070847Z InvocationID=7cb4a0ae-e99a-4fe6-8249-fb5bd8b29399 LogTimestamp=2021-05-28T00:17:56.070901Z Entering 2021-05-28T00:17:56.071Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=595b4720-cce9-4fa3-bb4b-5117245fc953 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:56.070847Z InvocationID=7cb4a0ae-e99a-4fe6-8249-fb5bd8b29399 LogTimestamp=2021-05-28T00:17:56.070901Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:17:56.190Z [qtp580718781-12] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:17:56.071Z InvocationID=df5f747a-cbb7-4759-9692-ed26900a8dfb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c198498', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 117ms 2021-05-28T00:17:56.191Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=119 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:17:56.071Z InvocationID=df5f747a-cbb7-4759-9692-ed26900a8dfb ErrorCode=409 LogTimestamp=2021-05-28T00:17:56.190963Z ResponseDesc=Conflict Exiting. 2021-05-28T00:17:56.191Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:17:56.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77df57a8-fef6-4e3c-ae41-016b4122bdf8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:17:56.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77df57a8-fef6-4e3c-ae41-016b4122bdf8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73c06712 2021-05-28T00:17:56.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77df57a8-fef6-4e3c-ae41-016b4122bdf8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:17:56.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77df57a8-fef6-4e3c-ae41-016b4122bdf8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:17:56.571Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=08e9bc51-d73d-45da-ae20-06475ea2d06d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:56.571079Z InvocationID=1ff8035e-21f5-4474-977c-3ad979e30a8c LogTimestamp=2021-05-28T00:17:56.571144Z Entering 2021-05-28T00:17:56.571Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=08e9bc51-d73d-45da-ae20-06475ea2d06d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:56.571079Z InvocationID=1ff8035e-21f5-4474-977c-3ad979e30a8c LogTimestamp=2021-05-28T00:17:56.571144Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:17:56.664Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=93 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:17:56.571Z InvocationID=8a3aa2c4-082b-4cf3-834c-e0b4a304fe41 LogTimestamp=2021-05-28T00:17:56.664327Z ResponseDesc=OK Exiting. 2021-05-28T00:17:56.664Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:17:56.775Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:56.775851Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:17:56.776Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:56.775851Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:56.776Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:17:56.874Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=15fbd5a7-103b-4744-bfff-0c217f0821d2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:56.873711Z InvocationID=52fb1b5e-ce64-41c1-8d2b-84fb4b9af1bc LogTimestamp=2021-05-28T00:17:56.873801Z Entering 2021-05-28T00:17:56.874Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=15fbd5a7-103b-4744-bfff-0c217f0821d2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:56.873711Z InvocationID=52fb1b5e-ce64-41c1-8d2b-84fb4b9af1bc LogTimestamp=2021-05-28T00:17:56.873801Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:17:56.970Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:17:56.874Z InvocationID=c05be18e-2de0-45d6-8611-1122bb8f2712 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:17:59.566Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2692 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:17:56.874Z InvocationID=c05be18e-2de0-45d6-8611-1122bb8f2712 LogTimestamp=2021-05-28T00:17:59.566445Z ResponseDesc=Created Exiting. 2021-05-28T00:17:59.567Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:17:59.704Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:59.704831Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:17:59.704Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:17:59.704831Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:17:59.705Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:17:59.781Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5b81916f-daef-4fca-bc3d-f4242e62a458 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:59.781212Z InvocationID=dbce0864-c8be-441d-9c41-008ec72af76c LogTimestamp=2021-05-28T00:17:59.781291Z Entering 2021-05-28T00:17:59.781Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5b81916f-daef-4fca-bc3d-f4242e62a458 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:17:59.781212Z InvocationID=dbce0864-c8be-441d-9c41-008ec72af76c LogTimestamp=2021-05-28T00:17:59.781291Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:17:59.895Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:17:59.781Z InvocationID=b81416b8-c1e1-48d2-82b6-4fbd6c07a102 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:18:00.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0797ff-e63e-48dc-beeb-9006378e151e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:00.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0797ff-e63e-48dc-beeb-9006378e151e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f162d4a 2021-05-28T00:18:00.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0797ff-e63e-48dc-beeb-9006378e151e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:00.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0797ff-e63e-48dc-beeb-9006378e151e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:02.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bb8ffc3-2a1c-4692-b939-76ff786ce0d6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:02.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bb8ffc3-2a1c-4692-b939-76ff786ce0d6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c3ca713 2021-05-28T00:18:02.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bb8ffc3-2a1c-4692-b939-76ff786ce0d6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:02.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bb8ffc3-2a1c-4692-b939-76ff786ce0d6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:02.601Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2820 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:17:59.781Z InvocationID=b81416b8-c1e1-48d2-82b6-4fbd6c07a102 LogTimestamp=2021-05-28T00:18:02.601467Z ResponseDesc=Created Exiting. 2021-05-28T00:18:02.602Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:02.677Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:02.677121Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:02.677Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:02.677121Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:02.686Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:18:02.865Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ee8001b3-a619-478b-bfd9-6667d4aed882 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:02.865092Z InvocationID=093b46a9-6546-4964-ab48-a4403f274892 LogTimestamp=2021-05-28T00:18:02.865146Z Entering 2021-05-28T00:18:02.865Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ee8001b3-a619-478b-bfd9-6667d4aed882 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:02.865092Z InvocationID=093b46a9-6546-4964-ab48-a4403f274892 LogTimestamp=2021-05-28T00:18:02.865146Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:02.891Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:02.865Z InvocationID=e19b6aa7-c563-4f8d-aa06-a5f4d73bc03e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:18:04.771Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1905 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:02.865Z InvocationID=e19b6aa7-c563-4f8d-aa06-a5f4d73bc03e LogTimestamp=2021-05-28T00:18:04.770630Z ResponseDesc=Created Exiting. 2021-05-28T00:18:04.771Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:04.870Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:04.870796Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:04.870Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:04.870796Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:04.871Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:18:04.976Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bbd47b45-01f0-4f68-8243-80f35e5a69c7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:04.975731Z InvocationID=d11ee0f9-acda-49f1-8640-5eeafd08f804 LogTimestamp=2021-05-28T00:18:04.975806Z Entering 2021-05-28T00:18:04.976Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bbd47b45-01f0-4f68-8243-80f35e5a69c7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:04.975731Z InvocationID=d11ee0f9-acda-49f1-8640-5eeafd08f804 LogTimestamp=2021-05-28T00:18:04.975806Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:05.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e99439-bf06-4381-a370-4ac250d1beda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:05.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e99439-bf06-4381-a370-4ac250d1beda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27af309e 2021-05-28T00:18:05.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e99439-bf06-4381-a370-4ac250d1beda PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:05.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8e99439-bf06-4381-a370-4ac250d1beda PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:05.075Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:04.976Z InvocationID=3f488e7a-e264-4e0d-acaf-11c7460fea77 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:18:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:18:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:18:06.979Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2003 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:04.976Z InvocationID=3f488e7a-e264-4e0d-acaf-11c7460fea77 LogTimestamp=2021-05-28T00:18:06.979128Z ResponseDesc=Created Exiting. 2021-05-28T00:18:06.980Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:07.070Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:07.070447Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:07.070Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:07.070447Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:07.071Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:18:07.084Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=93395cee-638a-4f98-b90c-e89ab575d137 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:07.083603Z InvocationID=e059eaa9-5940-46f6-9676-272130c1defc LogTimestamp=2021-05-28T00:18:07.083675Z Entering 2021-05-28T00:18:07.084Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=93395cee-638a-4f98-b90c-e89ab575d137 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:07.083603Z InvocationID=e059eaa9-5940-46f6-9676-272130c1defc LogTimestamp=2021-05-28T00:18:07.083675Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:07.174Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:07.084Z InvocationID=af184eb0-fe80-40af-a2c7-5b2f4eca2c24 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:18:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dba3487-e6eb-4f3d-9840-b90bf9503eb0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dba3487-e6eb-4f3d-9840-b90bf9503eb0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57272b8e 2021-05-28T00:18:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dba3487-e6eb-4f3d-9840-b90bf9503eb0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dba3487-e6eb-4f3d-9840-b90bf9503eb0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:08.681Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1596 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:07.084Z InvocationID=af184eb0-fe80-40af-a2c7-5b2f4eca2c24 LogTimestamp=2021-05-28T00:18:08.680714Z ResponseDesc=Created Exiting. 2021-05-28T00:18:08.681Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:08.766Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:08.766470Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:08.766Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:08.766470Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:08.767Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:18:08.780Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=394d8e8c-93b7-4592-90c0-d90b3a254e65 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:08.780665Z InvocationID=ca4baa44-0194-4105-9eeb-8fc72daba2ea LogTimestamp=2021-05-28T00:18:08.780710Z Entering 2021-05-28T00:18:08.781Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=394d8e8c-93b7-4592-90c0-d90b3a254e65 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:08.780665Z InvocationID=ca4baa44-0194-4105-9eeb-8fc72daba2ea LogTimestamp=2021-05-28T00:18:08.780710Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:08.875Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:08.781Z InvocationID=48decbe0-3438-4b2d-a6a1-18c32239d1ab PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:18:10.669Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1888 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:08.781Z InvocationID=48decbe0-3438-4b2d-a6a1-18c32239d1ab LogTimestamp=2021-05-28T00:18:10.669021Z ResponseDesc=Created Exiting. 2021-05-28T00:18:10.669Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:10.763Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:10.763782Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:10.763Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:10.763782Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:10.764Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:18:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25a4882-ed50-4a86-bb9f-078a83aa0147 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25a4882-ed50-4a86-bb9f-078a83aa0147 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49c5305d 2021-05-28T00:18:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25a4882-ed50-4a86-bb9f-078a83aa0147 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25a4882-ed50-4a86-bb9f-078a83aa0147 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:10.780Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b01f708f-28a5-470a-9ff1-210b4960fbc7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:10.779963Z InvocationID=224ce07c-0c06-4b95-8082-6def2b15de2a LogTimestamp=2021-05-28T00:18:10.780026Z Entering 2021-05-28T00:18:10.780Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b01f708f-28a5-470a-9ff1-210b4960fbc7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:10.779963Z InvocationID=224ce07c-0c06-4b95-8082-6def2b15de2a LogTimestamp=2021-05-28T00:18:10.780026Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:10.866Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:10.780Z InvocationID=bd837788-bfe5-41ad-a255-dd8813afe172 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:18:12.973Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:10.780Z InvocationID=bd837788-bfe5-41ad-a255-dd8813afe172 LogTimestamp=2021-05-28T00:18:12.972590Z ResponseDesc=Created Exiting. 2021-05-28T00:18:12.973Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:13.172Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:13.172850Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:13.173Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:13.172850Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:13.173Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:18:13.363Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7bd16e92-4b2c-4cc8-a8e8-3aeede139330 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:13.363238Z InvocationID=580737af-046b-4e09-a860-4753914995f9 LogTimestamp=2021-05-28T00:18:13.363335Z Entering 2021-05-28T00:18:13.363Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7bd16e92-4b2c-4cc8-a8e8-3aeede139330 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:13.363238Z InvocationID=580737af-046b-4e09-a860-4753914995f9 LogTimestamp=2021-05-28T00:18:13.363335Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:13.571Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:13.364Z InvocationID=0112d162-bae3-4622-a6c0-e0eacc33ee66 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:18:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb819e98-d897-4622-88f6-2ce0332e77db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb819e98-d897-4622-88f6-2ce0332e77db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@80249 2021-05-28T00:18:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb819e98-d897-4622-88f6-2ce0332e77db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb819e98-d897-4622-88f6-2ce0332e77db PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:15.370Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2005 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:13.364Z InvocationID=0112d162-bae3-4622-a6c0-e0eacc33ee66 LogTimestamp=2021-05-28T00:18:15.369778Z ResponseDesc=Created Exiting. 2021-05-28T00:18:15.370Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:15.470Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:15.470848Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:15.471Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:15.470848Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:15.471Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:18:15.481Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2bd85c65-e60a-4e35-97be-f1278cd00d62 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:15.481245Z InvocationID=0998cd0a-130d-49c8-9784-526b1562be1c LogTimestamp=2021-05-28T00:18:15.481292Z Entering 2021-05-28T00:18:15.481Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2bd85c65-e60a-4e35-97be-f1278cd00d62 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:15.481245Z InvocationID=0998cd0a-130d-49c8-9784-526b1562be1c LogTimestamp=2021-05-28T00:18:15.481292Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:15.571Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:15.481Z InvocationID=aed474de-ddf9-4776-a0b7-3981a92f83b8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:18:16.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22633dd7-9de4-4998-8c50-69730f2a72e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:16.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22633dd7-9de4-4998-8c50-69730f2a72e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5efdf916 2021-05-28T00:18:16.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22633dd7-9de4-4998-8c50-69730f2a72e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:16.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22633dd7-9de4-4998-8c50-69730f2a72e9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:17.559Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2078 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:15.481Z InvocationID=aed474de-ddf9-4776-a0b7-3981a92f83b8 LogTimestamp=2021-05-28T00:18:17.559407Z ResponseDesc=Created Exiting. 2021-05-28T00:18:17.560Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:17.661Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:17.661096Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:17.661Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:17.661096Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:17.661Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:18:17.672Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ade872ec-475e-483a-9521-beebf7683e3e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:17.672198Z InvocationID=481a8137-4d1b-4822-a2e2-5e312ec1610a LogTimestamp=2021-05-28T00:18:17.672260Z Entering 2021-05-28T00:18:17.672Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ade872ec-475e-483a-9521-beebf7683e3e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:17.672198Z InvocationID=481a8137-4d1b-4822-a2e2-5e312ec1610a LogTimestamp=2021-05-28T00:18:17.672260Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:17.764Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:17.672Z InvocationID=e85550e8-8e96-4a48-b119-155da9c20402 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:18:19.364Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1692 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:17.672Z InvocationID=e85550e8-8e96-4a48-b119-155da9c20402 LogTimestamp=2021-05-28T00:18:19.364397Z ResponseDesc=Created Exiting. 2021-05-28T00:18:19.365Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:19.470Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:19.470415Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:19.470Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:19.470415Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:19.470Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:18:19.574Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=38dbd46a-6e82-441f-9de7-e5f5da361bd5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:19.574101Z InvocationID=37684014-e7c8-4f87-bcf8-2cb81a75bba1 LogTimestamp=2021-05-28T00:18:19.574213Z Entering 2021-05-28T00:18:19.574Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=38dbd46a-6e82-441f-9de7-e5f5da361bd5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:19.574101Z InvocationID=37684014-e7c8-4f87-bcf8-2cb81a75bba1 LogTimestamp=2021-05-28T00:18:19.574213Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:19.674Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:19.574Z InvocationID=11476dd2-3091-49be-8f93-30c38260b0f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:18:19.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f72c02d-b76f-4f11-8703-5882b43482a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f72c02d-b76f-4f11-8703-5882b43482a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3068d21c 2021-05-28T00:18:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f72c02d-b76f-4f11-8703-5882b43482a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f72c02d-b76f-4f11-8703-5882b43482a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:18:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:18:21.762Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2187 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:19.574Z InvocationID=11476dd2-3091-49be-8f93-30c38260b0f1 LogTimestamp=2021-05-28T00:18:21.761618Z ResponseDesc=Created Exiting. 2021-05-28T00:18:21.762Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:21.782Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:21.782483Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:21.782Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:21.782483Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:21.783Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:18:21.863Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2c6927d2-7d51-4fd1-8b66-92b10f34426d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:21.862828Z InvocationID=2ef112a0-e3a4-4c1c-a9ab-23168b6eef87 LogTimestamp=2021-05-28T00:18:21.862870Z Entering 2021-05-28T00:18:21.863Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2c6927d2-7d51-4fd1-8b66-92b10f34426d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:21.862828Z InvocationID=2ef112a0-e3a4-4c1c-a9ab-23168b6eef87 LogTimestamp=2021-05-28T00:18:21.862870Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:21.872Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:21.863Z InvocationID=07151598-7e6c-4f81-bcf6-21881b3a18fe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:18:22.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=884cf507-4d1f-4946-b437-1edc2998157e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:22.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=884cf507-4d1f-4946-b437-1edc2998157e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@504f5b94 2021-05-28T00:18:22.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=884cf507-4d1f-4946-b437-1edc2998157e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:22.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=884cf507-4d1f-4946-b437-1edc2998157e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:24.061Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2198 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:21.863Z InvocationID=07151598-7e6c-4f81-bcf6-21881b3a18fe LogTimestamp=2021-05-28T00:18:24.061217Z ResponseDesc=Created Exiting. 2021-05-28T00:18:24.061Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:24.173Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:24.173196Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:24.173Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:24.173196Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:24.173Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:18:24.261Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6e8bfe8e-2874-4c93-b89c-19ccbb22fcaa InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:24.260799Z InvocationID=4c3ec9c3-fc3a-469f-b9c1-609339d1d8e7 LogTimestamp=2021-05-28T00:18:24.260854Z Entering 2021-05-28T00:18:24.261Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6e8bfe8e-2874-4c93-b89c-19ccbb22fcaa InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:24.260799Z InvocationID=4c3ec9c3-fc3a-469f-b9c1-609339d1d8e7 LogTimestamp=2021-05-28T00:18:24.260854Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:24.359Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:24.261Z InvocationID=777c0be5-319c-4352-aadc-e28daca6b25d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:18:26.360Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31121b1d-4dd5-4291-baa1-64de62a86f5b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:26.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31121b1d-4dd5-4291-baa1-64de62a86f5b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1095119e 2021-05-28T00:18:26.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31121b1d-4dd5-4291-baa1-64de62a86f5b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:26.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31121b1d-4dd5-4291-baa1-64de62a86f5b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:26.560Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2298 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:24.261Z InvocationID=777c0be5-319c-4352-aadc-e28daca6b25d LogTimestamp=2021-05-28T00:18:26.559989Z ResponseDesc=Created Exiting. 2021-05-28T00:18:26.561Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:26.574Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:26.574837Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:26.575Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:26.574837Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:26.575Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:18:26.674Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3804a2bd-2b8a-42b2-a72f-2c2802cdb220 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:26.673721Z InvocationID=d2f34404-0ca3-4ca4-ab8d-f41810a207e3 LogTimestamp=2021-05-28T00:18:26.673795Z Entering 2021-05-28T00:18:26.674Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3804a2bd-2b8a-42b2-a72f-2c2802cdb220 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:26.673721Z InvocationID=d2f34404-0ca3-4ca4-ab8d-f41810a207e3 LogTimestamp=2021-05-28T00:18:26.673795Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:26.774Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:26.674Z InvocationID=e8f6de91-ba05-485a-8984-ca168a23818a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:18:28.668Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1994 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:26.674Z InvocationID=e8f6de91-ba05-485a-8984-ca168a23818a LogTimestamp=2021-05-28T00:18:28.668385Z ResponseDesc=Created Exiting. 2021-05-28T00:18:28.669Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:28.775Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:28.775478Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:18:28.775Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:18:28.775478Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:28.776Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:18:28.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112b0e46-c7fb-42ad-8e00-c4e61d422613 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112b0e46-c7fb-42ad-8e00-c4e61d422613 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21116fd3 2021-05-28T00:18:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112b0e46-c7fb-42ad-8e00-c4e61d422613 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112b0e46-c7fb-42ad-8e00-c4e61d422613 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:28.872Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ab88a3cc-f703-492d-9954-6ea2af2cf85f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:28.872201Z InvocationID=00a10e96-4c1a-431a-8905-ea073bcade71 LogTimestamp=2021-05-28T00:18:28.872246Z Entering 2021-05-28T00:18:28.873Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ab88a3cc-f703-492d-9954-6ea2af2cf85f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:28.872201Z InvocationID=00a10e96-4c1a-431a-8905-ea073bcade71 LogTimestamp=2021-05-28T00:18:28.872246Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:18:28.972Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:28.873Z InvocationID=9c536b79-dfcb-426f-a413-365c4c1fdd68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:18:31.268Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2394 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:18:28.873Z InvocationID=9c536b79-dfcb-426f-a413-365c4c1fdd68 LogTimestamp=2021-05-28T00:18:31.267803Z ResponseDesc=Created Exiting. 2021-05-28T00:18:31.268Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:18:31.761Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5e434f13-750e-4cc2-b4b1-8d01e76efa2a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:31.761070Z InvocationID=7fe855db-da7e-4764-acc6-033948174177 LogTimestamp=2021-05-28T00:18:31.761134Z Entering 2021-05-28T00:18:31.761Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5e434f13-750e-4cc2-b4b1-8d01e76efa2a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:31.761070Z InvocationID=7fe855db-da7e-4764-acc6-033948174177 LogTimestamp=2021-05-28T00:18:31.761134Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:18:31.873Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=112 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:18:31.761Z InvocationID=e5eef07a-b04b-43a3-809a-1e6ce7ddb746 LogTimestamp=2021-05-28T00:18:31.873188Z ResponseDesc=OK Exiting. 2021-05-28T00:18:31.873Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:18:32.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcfd47-0c57-487d-ba9e-d1c2bd081849 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:32.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcfd47-0c57-487d-ba9e-d1c2bd081849 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@571fb1df 2021-05-28T00:18:32.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcfd47-0c57-487d-ba9e-d1c2bd081849 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:32.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3fcfd47-0c57-487d-ba9e-d1c2bd081849 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:32.365Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=159efbde-9cd6-4c55-b42d-d212d1f40903 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:32.365053Z InvocationID=b306a399-52ac-486a-8235-a0e0f05241f6 LogTimestamp=2021-05-28T00:18:32.365097Z Entering 2021-05-28T00:18:32.365Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=159efbde-9cd6-4c55-b42d-d212d1f40903 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:32.365053Z InvocationID=b306a399-52ac-486a-8235-a0e0f05241f6 LogTimestamp=2021-05-28T00:18:32.365097Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:18:32.766Z [qtp580718781-18] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:18:32.365Z InvocationID=81802e26-b7bf-41c1-9057-58da682a2389 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 398987862 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n398987862 100% \n 2021-05-28T00:18:32.774Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=408 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:18:32.365Z InvocationID=81802e26-b7bf-41c1-9057-58da682a2389 LogTimestamp=2021-05-28T00:18:32.773792Z ResponseDesc=OK Exiting. 2021-05-28T00:18:32.774Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:18:32.865Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2249718b-ab5d-4670-a3ac-605c2590bde3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:32.865104Z InvocationID=3284f64b-9aca-44a6-b977-0cea228a96c8 LogTimestamp=2021-05-28T00:18:32.865149Z Entering 2021-05-28T00:18:32.865Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2249718b-ab5d-4670-a3ac-605c2590bde3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:32.865104Z InvocationID=3284f64b-9aca-44a6-b977-0cea228a96c8 LogTimestamp=2021-05-28T00:18:32.865149Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:18:32.964Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:18:32.865Z InvocationID=0d0a278c-34aa-42fb-bdc0-5476f679eb1b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:18:32.970Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:18:32.865Z InvocationID=0d0a278c-34aa-42fb-bdc0-5476f679eb1b ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:18:32.971Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:18:32.865Z InvocationID=0d0a278c-34aa-42fb-bdc0-5476f679eb1b ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:18:32.972Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=106 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:18:32.865Z InvocationID=0d0a278c-34aa-42fb-bdc0-5476f679eb1b ErrorCode=500 LogTimestamp=2021-05-28T00:18:32.971817Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:32.972Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:18:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f4990e5-7905-4288-9954-869a51e7790b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f4990e5-7905-4288-9954-869a51e7790b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d26d353 2021-05-28T00:18:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f4990e5-7905-4288-9954-869a51e7790b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f4990e5-7905-4288-9954-869a51e7790b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:18:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:18:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:18:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:18:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:18:38.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f31342c-3a5d-452c-8971-0c38ccc3be77 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:38.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f31342c-3a5d-452c-8971-0c38ccc3be77 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45873488 2021-05-28T00:18:38.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f31342c-3a5d-452c-8971-0c38ccc3be77 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:38.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f31342c-3a5d-452c-8971-0c38ccc3be77 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:40.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f032b29-2f8d-4ac3-80de-82a3cb24d994 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:40.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f032b29-2f8d-4ac3-80de-82a3cb24d994 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fc2a41c 2021-05-28T00:18:40.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f032b29-2f8d-4ac3-80de-82a3cb24d994 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:40.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f032b29-2f8d-4ac3-80de-82a3cb24d994 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:43.174Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=579df102-d726-46d2-8968-2c1c8e4972f8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:43.173858Z InvocationID=cebec824-ef51-4659-99af-70e6cf5081be LogTimestamp=2021-05-28T00:18:43.173910Z Entering 2021-05-28T00:18:43.174Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=579df102-d726-46d2-8968-2c1c8e4972f8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:43.173858Z InvocationID=cebec824-ef51-4659-99af-70e6cf5081be LogTimestamp=2021-05-28T00:18:43.173910Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:18:43.269Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:18:43.174Z InvocationID=a7bd9fc4-3b8b-4703-935a-1643a88a98b7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:18:43.458Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:18:43.174Z InvocationID=a7bd9fc4-3b8b-4703-935a-1643a88a98b7 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:18:43.459Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:18:43.174Z InvocationID=a7bd9fc4-3b8b-4703-935a-1643a88a98b7 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:18:43.460Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=285 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:18:43.174Z InvocationID=a7bd9fc4-3b8b-4703-935a-1643a88a98b7 ErrorCode=500 LogTimestamp=2021-05-28T00:18:43.459962Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:43.460Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:18:43.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=258b914b-2914-47ce-b63e-aa9ba7885333 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:43.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=258b914b-2914-47ce-b63e-aa9ba7885333 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45f97172 2021-05-28T00:18:43.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=258b914b-2914-47ce-b63e-aa9ba7885333 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:43.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=258b914b-2914-47ce-b63e-aa9ba7885333 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:46.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78d4bb5b-37a5-472d-8371-5efa9a745e61 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:46.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78d4bb5b-37a5-472d-8371-5efa9a745e61 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e8e9ed5 2021-05-28T00:18:46.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78d4bb5b-37a5-472d-8371-5efa9a745e61 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:46.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78d4bb5b-37a5-472d-8371-5efa9a745e61 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64714262-a696-4403-a768-0fc4f7eff295 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64714262-a696-4403-a768-0fc4f7eff295 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72d1c8f8 2021-05-28T00:18:49.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64714262-a696-4403-a768-0fc4f7eff295 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:49.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64714262-a696-4403-a768-0fc4f7eff295 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:18:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:18:52.224Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:18:52.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=436d7717-6f83-4e27-afc8-5f1e55c83ee0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:52.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=436d7717-6f83-4e27-afc8-5f1e55c83ee0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@84960fa 2021-05-28T00:18:52.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=436d7717-6f83-4e27-afc8-5f1e55c83ee0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:52.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=436d7717-6f83-4e27-afc8-5f1e55c83ee0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:53.574Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=64002c7a-fbce-44c9-a547-3c80ec500817 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:53.573917Z InvocationID=428893be-26b4-4188-8c38-89444304140f LogTimestamp=2021-05-28T00:18:53.574006Z Entering 2021-05-28T00:18:53.574Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=64002c7a-fbce-44c9-a547-3c80ec500817 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:18:53.573917Z InvocationID=428893be-26b4-4188-8c38-89444304140f LogTimestamp=2021-05-28T00:18:53.574006Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:18:53.679Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:18:53.575Z InvocationID=b5df2626-ec1b-4927-bc29-18a721d80b9d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:18:53.768Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:18:53.575Z InvocationID=b5df2626-ec1b-4927-bc29-18a721d80b9d ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:18:53.769Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:18:53.575Z InvocationID=b5df2626-ec1b-4927-bc29-18a721d80b9d ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:18:53.769Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=194 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:18:53.575Z InvocationID=b5df2626-ec1b-4927-bc29-18a721d80b9d ErrorCode=500 LogTimestamp=2021-05-28T00:18:53.769703Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:18:53.770Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:18:56.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f76b3418-cb20-4bf2-9293-7d16fcc57a50 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:56.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f76b3418-cb20-4bf2-9293-7d16fcc57a50 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@635f260 2021-05-28T00:18:56.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f76b3418-cb20-4bf2-9293-7d16fcc57a50 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:56.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f76b3418-cb20-4bf2-9293-7d16fcc57a50 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:18:58.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a0b048-1780-4763-8ade-a304a1da0d8b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:18:58.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a0b048-1780-4763-8ade-a304a1da0d8b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c37bc45 2021-05-28T00:18:58.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a0b048-1780-4763-8ade-a304a1da0d8b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:18:58.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a0b048-1780-4763-8ade-a304a1da0d8b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:01.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f03429-1d5e-4e0c-842d-da8004d7c4a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:01.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f03429-1d5e-4e0c-842d-da8004d7c4a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d2d33 2021-05-28T00:19:01.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f03429-1d5e-4e0c-842d-da8004d7c4a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:01.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f03429-1d5e-4e0c-842d-da8004d7c4a3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:03.963Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a0059954-4b30-4efc-b17b-6660fb268136 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:03.963306Z InvocationID=c9095129-252f-482c-addf-3867334b1822 LogTimestamp=2021-05-28T00:19:03.963400Z Entering 2021-05-28T00:19:03.963Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a0059954-4b30-4efc-b17b-6660fb268136 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:03.963306Z InvocationID=c9095129-252f-482c-addf-3867334b1822 LogTimestamp=2021-05-28T00:19:03.963400Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:19:04.071Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:19:03.964Z InvocationID=4b7527d6-9dab-4139-be9a-cbf1cb0afd2c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:19:04.267Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:03.964Z InvocationID=4b7527d6-9dab-4139-be9a-cbf1cb0afd2c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:19:04.268Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:03.964Z InvocationID=4b7527d6-9dab-4139-be9a-cbf1cb0afd2c ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:19:04.269Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=304 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:19:03.964Z InvocationID=4b7527d6-9dab-4139-be9a-cbf1cb0afd2c ErrorCode=500 LogTimestamp=2021-05-28T00:19:04.268913Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:19:04.269Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:19:04.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=918284b4-9896-41f4-a7a9-cea971770e4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:04.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=918284b4-9896-41f4-a7a9-cea971770e4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4350c372 2021-05-28T00:19:04.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=918284b4-9896-41f4-a7a9-cea971770e4d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:04.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=918284b4-9896-41f4-a7a9-cea971770e4d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:19:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:19:07.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e63ab23-bcd2-42ae-9a29-f9f1b081736e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:07.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e63ab23-bcd2-42ae-9a29-f9f1b081736e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a5f8f8a 2021-05-28T00:19:07.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e63ab23-bcd2-42ae-9a29-f9f1b081736e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:07.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e63ab23-bcd2-42ae-9a29-f9f1b081736e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:10.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b042d8-7358-4be7-8160-cf45e332ad90 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:10.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b042d8-7358-4be7-8160-cf45e332ad90 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b4bc310 2021-05-28T00:19:10.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b042d8-7358-4be7-8160-cf45e332ad90 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b042d8-7358-4be7-8160-cf45e332ad90 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:14.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a40b46ee-8611-42c4-bdca-ec89f5360bf9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:14.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a40b46ee-8611-42c4-bdca-ec89f5360bf9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@421657ff 2021-05-28T00:19:14.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a40b46ee-8611-42c4-bdca-ec89f5360bf9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:14.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a40b46ee-8611-42c4-bdca-ec89f5360bf9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:14.570Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3de5bbd6-be00-4582-bc16-723e7805c498 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:14.569495Z InvocationID=e12b05e7-c69d-4462-84fd-ac169c4a2c3a LogTimestamp=2021-05-28T00:19:14.569666Z Entering 2021-05-28T00:19:14.570Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3de5bbd6-be00-4582-bc16-723e7805c498 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:14.569495Z InvocationID=e12b05e7-c69d-4462-84fd-ac169c4a2c3a LogTimestamp=2021-05-28T00:19:14.569666Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:19:14.674Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:19:14.570Z InvocationID=f3aed5e2-2b31-4df9-9a30-9e843b067cdd PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:19:14.859Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:14.570Z InvocationID=f3aed5e2-2b31-4df9-9a30-9e843b067cdd ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:19:14.859Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:14.570Z InvocationID=f3aed5e2-2b31-4df9-9a30-9e843b067cdd ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:19:14.860Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=290 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:19:14.570Z InvocationID=f3aed5e2-2b31-4df9-9a30-9e843b067cdd ErrorCode=500 LogTimestamp=2021-05-28T00:19:14.860068Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:19:14.860Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:19:16.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e86355fd-40d9-4e78-9fff-a32401c3fbe3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:16.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e86355fd-40d9-4e78-9fff-a32401c3fbe3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@718579c 2021-05-28T00:19:16.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e86355fd-40d9-4e78-9fff-a32401c3fbe3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:16.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e86355fd-40d9-4e78-9fff-a32401c3fbe3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:19.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe3e427d-da6c-40cf-807b-f890bd189e04 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:19.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe3e427d-da6c-40cf-807b-f890bd189e04 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11adbb2a 2021-05-28T00:19:19.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe3e427d-da6c-40cf-807b-f890bd189e04 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:19.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe3e427d-da6c-40cf-807b-f890bd189e04 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:19:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:19:22.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=921b9284-fad5-48cb-8ae9-630eadb72e2d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:22.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=921b9284-fad5-48cb-8ae9-630eadb72e2d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@640ba94c 2021-05-28T00:19:22.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=921b9284-fad5-48cb-8ae9-630eadb72e2d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:22.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=921b9284-fad5-48cb-8ae9-630eadb72e2d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:24.987Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=89d6c762-7454-4761-979e-6c35be89b7a5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:24.986678Z InvocationID=3794f747-f041-42a8-b5cf-d3c5ec4acb3e LogTimestamp=2021-05-28T00:19:24.986753Z Entering 2021-05-28T00:19:24.987Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=89d6c762-7454-4761-979e-6c35be89b7a5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:24.986678Z InvocationID=3794f747-f041-42a8-b5cf-d3c5ec4acb3e LogTimestamp=2021-05-28T00:19:24.986753Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:19:25.079Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:19:24.987Z InvocationID=b328eece-6d15-4006-ae6e-c396092595f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:19:25.479Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:24.987Z InvocationID=b328eece-6d15-4006-ae6e-c396092595f0 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:19:25.480Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:24.987Z InvocationID=b328eece-6d15-4006-ae6e-c396092595f0 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:19:25.480Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=493 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:19:24.987Z InvocationID=b328eece-6d15-4006-ae6e-c396092595f0 ErrorCode=500 LogTimestamp=2021-05-28T00:19:25.480515Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:19:25.481Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:19:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35f456-6dac-4623-bab7-70fe693ad804 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35f456-6dac-4623-bab7-70fe693ad804 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14d2b39d 2021-05-28T00:19:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35f456-6dac-4623-bab7-70fe693ad804 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:25.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35f456-6dac-4623-bab7-70fe693ad804 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:28.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a5ea133-9d36-49de-ade2-fa0d0468f61c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:28.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a5ea133-9d36-49de-ade2-fa0d0468f61c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5133b22c 2021-05-28T00:19:28.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a5ea133-9d36-49de-ade2-fa0d0468f61c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:28.795Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a5ea133-9d36-49de-ade2-fa0d0468f61c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:31.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a86095be-7ecb-4dd3-98d2-c16a7bd61e79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:31.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a86095be-7ecb-4dd3-98d2-c16a7bd61e79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d3b37cb 2021-05-28T00:19:31.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a86095be-7ecb-4dd3-98d2-c16a7bd61e79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:31.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a86095be-7ecb-4dd3-98d2-c16a7bd61e79 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:34.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65ba2316-16e2-40e4-a40b-9e8d6d948347 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:34.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65ba2316-16e2-40e4-a40b-9e8d6d948347 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14c3aae9 2021-05-28T00:19:34.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65ba2316-16e2-40e4-a40b-9e8d6d948347 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:34.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65ba2316-16e2-40e4-a40b-9e8d6d948347 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:19:35.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:19:35.872Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7d57bfb8-b192-4de3-aea2-eaff2055da07 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:35.872005Z InvocationID=00fb160e-a904-42af-9845-fece412b052d LogTimestamp=2021-05-28T00:19:35.872127Z Entering 2021-05-28T00:19:35.872Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7d57bfb8-b192-4de3-aea2-eaff2055da07 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:35.872005Z InvocationID=00fb160e-a904-42af-9845-fece412b052d LogTimestamp=2021-05-28T00:19:35.872127Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:19:35.973Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:19:35.873Z InvocationID=02f99f62-8b15-4dce-9d2e-27e1da38d7de PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:19:36.156Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:35.873Z InvocationID=02f99f62-8b15-4dce-9d2e-27e1da38d7de ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:19:36.156Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:35.873Z InvocationID=02f99f62-8b15-4dce-9d2e-27e1da38d7de ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:19:36.157Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=284 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:19:35.873Z InvocationID=02f99f62-8b15-4dce-9d2e-27e1da38d7de ErrorCode=500 LogTimestamp=2021-05-28T00:19:36.157213Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:19:36.157Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:19:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:19:37.165Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:19:37.165Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:19:37.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5fc75f9-cf64-4be6-9f4d-456d76077e71 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:37.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5fc75f9-cf64-4be6-9f4d-456d76077e71 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@158cbdb7 2021-05-28T00:19:37.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5fc75f9-cf64-4be6-9f4d-456d76077e71 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:37.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5fc75f9-cf64-4be6-9f4d-456d76077e71 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:40.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ea10162-977e-4698-a68e-5a22a3b7ad0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:40.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ea10162-977e-4698-a68e-5a22a3b7ad0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5962cff8 2021-05-28T00:19:40.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ea10162-977e-4698-a68e-5a22a3b7ad0d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:40.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ea10162-977e-4698-a68e-5a22a3b7ad0d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:44.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1be7cb64-fab8-4c38-a5fc-83b6304e9e17 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:44.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1be7cb64-fab8-4c38-a5fc-83b6304e9e17 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d932e70 2021-05-28T00:19:44.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1be7cb64-fab8-4c38-a5fc-83b6304e9e17 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:44.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1be7cb64-fab8-4c38-a5fc-83b6304e9e17 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:46.263Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2f9a4578-a4f9-4c4b-b29e-1d002d1d57d2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:46.263438Z InvocationID=56b41c4c-d92e-48f4-a9af-ad63529d4395 LogTimestamp=2021-05-28T00:19:46.263532Z Entering 2021-05-28T00:19:46.264Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2f9a4578-a4f9-4c4b-b29e-1d002d1d57d2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:46.263438Z InvocationID=56b41c4c-d92e-48f4-a9af-ad63529d4395 LogTimestamp=2021-05-28T00:19:46.263532Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:19:46.355Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:19:46.264Z InvocationID=2e6ecc8c-e12d-4dae-9d6b-35afc7c0616b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:19:46.363Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:46.264Z InvocationID=2e6ecc8c-e12d-4dae-9d6b-35afc7c0616b ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:19:46.363Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:46.264Z InvocationID=2e6ecc8c-e12d-4dae-9d6b-35afc7c0616b ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:19:46.364Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=100 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:19:46.264Z InvocationID=2e6ecc8c-e12d-4dae-9d6b-35afc7c0616b ErrorCode=500 LogTimestamp=2021-05-28T00:19:46.364073Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:19:46.364Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:19:46.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da7e3a0e-9b59-47af-9330-adde320b5f11 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:46.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da7e3a0e-9b59-47af-9330-adde320b5f11 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@240e5dc9 2021-05-28T00:19:46.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da7e3a0e-9b59-47af-9330-adde320b5f11 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:46.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da7e3a0e-9b59-47af-9330-adde320b5f11 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:49.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ee4b0f6-7d5c-4265-b563-3be3dab1ac79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:49.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ee4b0f6-7d5c-4265-b563-3be3dab1ac79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@541f3dfa 2021-05-28T00:19:49.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ee4b0f6-7d5c-4265-b563-3be3dab1ac79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:49.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ee4b0f6-7d5c-4265-b563-3be3dab1ac79 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:19:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:19:52.150Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:19:52.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7217cc9-72d1-4c91-8667-4be37002b47e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:52.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7217cc9-72d1-4c91-8667-4be37002b47e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b4d258e 2021-05-28T00:19:52.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7217cc9-72d1-4c91-8667-4be37002b47e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:52.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7217cc9-72d1-4c91-8667-4be37002b47e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:55.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=603e8068-56bb-4958-9db2-aca0f3d5ab83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:55.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=603e8068-56bb-4958-9db2-aca0f3d5ab83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f196839 2021-05-28T00:19:55.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=603e8068-56bb-4958-9db2-aca0f3d5ab83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:55.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=603e8068-56bb-4958-9db2-aca0f3d5ab83 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:19:57.785Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=860e961c-5d2d-4ee7-803c-f113cfb174f8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=1 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:57.782762Z InvocationID=c05d1048-f307-4cd9-adf6-7b6db1da71ce LogTimestamp=2021-05-28T00:19:57.784667Z Entering 2021-05-28T00:19:57.785Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=860e961c-5d2d-4ee7-803c-f113cfb174f8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=1 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:19:57.782762Z InvocationID=c05d1048-f307-4cd9-adf6-7b6db1da71ce LogTimestamp=2021-05-28T00:19:57.784667Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:19:57.879Z [qtp580718781-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:19:57.785Z InvocationID=ab7da5af-cfea-4a97-80ae-6465b41f16ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:19:57.969Z [qtp580718781-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:57.785Z InvocationID=ab7da5af-cfea-4a97-80ae-6465b41f16ed ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:19:57.970Z [qtp580718781-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:19:57.785Z InvocationID=ab7da5af-cfea-4a97-80ae-6465b41f16ed ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:19:57.971Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=185 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:19:57.785Z InvocationID=ab7da5af-cfea-4a97-80ae-6465b41f16ed ErrorCode=500 LogTimestamp=2021-05-28T00:19:57.970912Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:19:57.971Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:19:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d5e50c9-e887-4681-8f2e-528d8f2a8199 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:19:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d5e50c9-e887-4681-8f2e-528d8f2a8199 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79af8806 2021-05-28T00:19:58.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d5e50c9-e887-4681-8f2e-528d8f2a8199 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:19:58.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d5e50c9-e887-4681-8f2e-528d8f2a8199 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12df90ab-ed51-44fc-a64d-e503ecee6558 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12df90ab-ed51-44fc-a64d-e503ecee6558 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54f28a24 2021-05-28T00:20:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12df90ab-ed51-44fc-a64d-e503ecee6558 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12df90ab-ed51-44fc-a64d-e503ecee6558 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:05.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe922ce-bf72-495b-ab62-8fae0058d484 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:05.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe922ce-bf72-495b-ab62-8fae0058d484 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26e1b461 2021-05-28T00:20:05.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe922ce-bf72-495b-ab62-8fae0058d484 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:05.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efe922ce-bf72-495b-ab62-8fae0058d484 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:20:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:20:08.285Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8db31f57-25ad-4e4f-a5f8-670b1c602fe5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:20:08.284403Z InvocationID=6d79962e-3ca3-479c-bb7a-651912b70259 LogTimestamp=2021-05-28T00:20:08.284604Z Entering 2021-05-28T00:20:08.285Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8db31f57-25ad-4e4f-a5f8-670b1c602fe5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:20:08.284403Z InvocationID=6d79962e-3ca3-479c-bb7a-651912b70259 LogTimestamp=2021-05-28T00:20:08.284604Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:20:08.395Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:20:08.285Z InvocationID=cf1b5af8-acf1-4278-87b1-2924f9734439 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:20:08.483Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:20:08.285Z InvocationID=cf1b5af8-acf1-4278-87b1-2924f9734439 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:20:08.483Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:20:08.285Z InvocationID=cf1b5af8-acf1-4278-87b1-2924f9734439 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:20:08.484Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=198 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:20:08.285Z InvocationID=cf1b5af8-acf1-4278-87b1-2924f9734439 ErrorCode=500 LogTimestamp=2021-05-28T00:20:08.483990Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:20:08.484Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:20:08.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8532447a-2aa1-4cf7-9ab9-0cb40cef16b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:08.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8532447a-2aa1-4cf7-9ab9-0cb40cef16b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42dd4824 2021-05-28T00:20:08.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8532447a-2aa1-4cf7-9ab9-0cb40cef16b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:08.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8532447a-2aa1-4cf7-9ab9-0cb40cef16b8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:11.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5706a56-61e0-436d-a340-9e288405b5fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:11.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5706a56-61e0-436d-a340-9e288405b5fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47b717c1 2021-05-28T00:20:11.082Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5706a56-61e0-436d-a340-9e288405b5fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:11.082Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5706a56-61e0-436d-a340-9e288405b5fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:14.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4dcb127-7dc8-4c9c-b913-42c918aa9dea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:14.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4dcb127-7dc8-4c9c-b913-42c918aa9dea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d406b9 2021-05-28T00:20:14.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4dcb127-7dc8-4c9c-b913-42c918aa9dea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:14.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4dcb127-7dc8-4c9c-b913-42c918aa9dea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:17.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e30f1ad4-dd99-466d-8b69-1003cf85b1fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:17.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e30f1ad4-dd99-466d-8b69-1003cf85b1fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cbdf89e 2021-05-28T00:20:17.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e30f1ad4-dd99-466d-8b69-1003cf85b1fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:17.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e30f1ad4-dd99-466d-8b69-1003cf85b1fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:18.678Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=edf45737-6757-4543-a2d8-bd125db2df63 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:20:18.678580Z InvocationID=5d5944c7-3e00-4634-9991-6be54bef276b LogTimestamp=2021-05-28T00:20:18.678644Z Entering 2021-05-28T00:20:18.679Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=edf45737-6757-4543-a2d8-bd125db2df63 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.176 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.176 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:20:18.678580Z InvocationID=5d5944c7-3e00-4634-9991-6be54bef276b LogTimestamp=2021-05-28T00:20:18.678644Z localAddr=10.242.102.221 remoteAddr=10.242.170.176 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:20:18.778Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:20:18.679Z InvocationID=1f73e8f8-9c34-4f22-bb0e-e4bf5b57da51 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:20:18.872Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:20:18.679Z InvocationID=1f73e8f8-9c34-4f22-bb0e-e4bf5b57da51 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:20:18.872Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:20:18.679Z InvocationID=1f73e8f8-9c34-4f22-bb0e-e4bf5b57da51 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:20:18.873Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=194 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:20:18.679Z InvocationID=1f73e8f8-9c34-4f22-bb0e-e4bf5b57da51 ErrorCode=500 LogTimestamp=2021-05-28T00:20:18.873476Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:20:18.874Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:20:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:20:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:20:21.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829fee8f-f0bc-477c-9319-dd80cb73e783 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:21.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829fee8f-f0bc-477c-9319-dd80cb73e783 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4890945 2021-05-28T00:20:21.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829fee8f-f0bc-477c-9319-dd80cb73e783 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:21.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829fee8f-f0bc-477c-9319-dd80cb73e783 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:23.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e29ae9-caa4-4ceb-9889-a9c3504e8a51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:23.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e29ae9-caa4-4ceb-9889-a9c3504e8a51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bff0f0a 2021-05-28T00:20:23.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e29ae9-caa4-4ceb-9889-a9c3504e8a51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:23.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96e29ae9-caa4-4ceb-9889-a9c3504e8a51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a5c66a-2a33-4c05-b5a5-594e7d829bd6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a5c66a-2a33-4c05-b5a5-594e7d829bd6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cb93f02 2021-05-28T00:20:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a5c66a-2a33-4c05-b5a5-594e7d829bd6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a5c66a-2a33-4c05-b5a5-594e7d829bd6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee83e028-25c6-4df3-9c4e-75f0de3c4611 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee83e028-25c6-4df3-9c4e-75f0de3c4611 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f034503 2021-05-28T00:20:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee83e028-25c6-4df3-9c4e-75f0de3c4611 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee83e028-25c6-4df3-9c4e-75f0de3c4611 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:31.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf42346d-124c-4d7c-b45f-b4c84dc726c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:31.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf42346d-124c-4d7c-b45f-b4c84dc726c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@530cb487 2021-05-28T00:20:31.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf42346d-124c-4d7c-b45f-b4c84dc726c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:31.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf42346d-124c-4d7c-b45f-b4c84dc726c4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:35.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88543811-64bb-4367-a5e9-454bb20287aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:35.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88543811-64bb-4367-a5e9-454bb20287aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@508866b3 2021-05-28T00:20:35.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88543811-64bb-4367-a5e9-454bb20287aa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:35.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88543811-64bb-4367-a5e9-454bb20287aa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:20:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:20:37.164Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:20:37.166Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:20:37.166Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:20:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bf240e-b597-42fc-b5d8-d629af77b5e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bf240e-b597-42fc-b5d8-d629af77b5e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@455c028f 2021-05-28T00:20:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bf240e-b597-42fc-b5d8-d629af77b5e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bf240e-b597-42fc-b5d8-d629af77b5e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:20:41.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:20:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:20:45.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31dd7689-3918-4c46-9d55-e9719c3cdb93 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:45.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31dd7689-3918-4c46-9d55-e9719c3cdb93 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40f26c7a 2021-05-28T00:20:45.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31dd7689-3918-4c46-9d55-e9719c3cdb93 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:45.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31dd7689-3918-4c46-9d55-e9719c3cdb93 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:46.679Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:20:48.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=443421f6-5bb9-4bf5-a460-c56e1e2606c9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:48.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=443421f6-5bb9-4bf5-a460-c56e1e2606c9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@766c406f 2021-05-28T00:20:48.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=443421f6-5bb9-4bf5-a460-c56e1e2606c9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:48.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=443421f6-5bb9-4bf5-a460-c56e1e2606c9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:48.974Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=443421f6-5bb9-4bf5-a460-c56e1e2606c9 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:20:48.975Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=443421f6-5bb9-4bf5-a460-c56e1e2606c9 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:20:49.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a6cf16f-02c4-41f5-aab4-fe66bc7faaf7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:49.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a6cf16f-02c4-41f5-aab4-fe66bc7faaf7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65c20def 2021-05-28T00:20:49.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a6cf16f-02c4-41f5-aab4-fe66bc7faaf7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:49.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a6cf16f-02c4-41f5-aab4-fe66bc7faaf7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:50.374Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7a6cf16f-02c4-41f5-aab4-fe66bc7faaf7 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:20:50.374Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7a6cf16f-02c4-41f5-aab4-fe66bc7faaf7 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:20:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:20:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:20:51.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f129ac7-06fd-4f91-9327-7c56e055dc35 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:20:51.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f129ac7-06fd-4f91-9327-7c56e055dc35 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40af33a2 2021-05-28T00:20:51.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f129ac7-06fd-4f91-9327-7c56e055dc35 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:20:51.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f129ac7-06fd-4f91-9327-7c56e055dc35 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:20:52.116Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:20:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:20:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:20:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:20:57.059Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:20:57.071Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:20:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:21:02.362Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:21:02.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78bf2d95-6255-4280-96ef-4b9f792ec1dc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:02.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78bf2d95-6255-4280-96ef-4b9f792ec1dc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13ed1671 2021-05-28T00:21:02.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78bf2d95-6255-4280-96ef-4b9f792ec1dc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:02.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78bf2d95-6255-4280-96ef-4b9f792ec1dc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:05.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5af834-f4c4-4bf4-bc4e-dc316394e99a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:05.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5af834-f4c4-4bf4-bc4e-dc316394e99a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@796535b 2021-05-28T00:21:05.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5af834-f4c4-4bf4-bc4e-dc316394e99a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:05.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5af834-f4c4-4bf4-bc4e-dc316394e99a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:21:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:21:06.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19dafd51-2c46-40f0-9313-60f833e725cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:06.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19dafd51-2c46-40f0-9313-60f833e725cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@724e9b33 2021-05-28T00:21:06.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19dafd51-2c46-40f0-9313-60f833e725cd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:06.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19dafd51-2c46-40f0-9313-60f833e725cd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:08.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e2a3de-f52b-44cd-b22e-a9cccbe3a097 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:08.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e2a3de-f52b-44cd-b22e-a9cccbe3a097 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b4594fb 2021-05-28T00:21:08.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e2a3de-f52b-44cd-b22e-a9cccbe3a097 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:08.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e2a3de-f52b-44cd-b22e-a9cccbe3a097 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:16.385Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd1af591-0130-4631-9872-2577bf5ee9ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:16.385Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd1af591-0130-4631-9872-2577bf5ee9ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30e04d7c 2021-05-28T00:21:16.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd1af591-0130-4631-9872-2577bf5ee9ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:16.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd1af591-0130-4631-9872-2577bf5ee9ad PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb4d6a1e-0188-4acb-be97-b79693dfd28a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb4d6a1e-0188-4acb-be97-b79693dfd28a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d9ccb0 2021-05-28T00:21:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb4d6a1e-0188-4acb-be97-b79693dfd28a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb4d6a1e-0188-4acb-be97-b79693dfd28a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:20.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=755f76d6-6eda-4e0a-8ed2-85917a6632dd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:20.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=755f76d6-6eda-4e0a-8ed2-85917a6632dd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5674ccbd 2021-05-28T00:21:20.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=755f76d6-6eda-4e0a-8ed2-85917a6632dd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:20.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=755f76d6-6eda-4e0a-8ed2-85917a6632dd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:21:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:21:21.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0c13661-685b-4ff1-b153-f35ac74bbc9b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:21.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0c13661-685b-4ff1-b153-f35ac74bbc9b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d03368f 2021-05-28T00:21:21.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0c13661-685b-4ff1-b153-f35ac74bbc9b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:21.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0c13661-685b-4ff1-b153-f35ac74bbc9b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:24.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c6b1de9-81e2-48f9-af11-510086e46564 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:24.059Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c6b1de9-81e2-48f9-af11-510086e46564 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ccbac94 2021-05-28T00:21:24.059Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c6b1de9-81e2-48f9-af11-510086e46564 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:24.059Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c6b1de9-81e2-48f9-af11-510086e46564 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab548821-b71d-4087-a438-908f9b880c9e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab548821-b71d-4087-a438-908f9b880c9e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@573a13b6 2021-05-28T00:21:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab548821-b71d-4087-a438-908f9b880c9e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:25.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab548821-b71d-4087-a438-908f9b880c9e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:27.457Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03144841-47cf-4105-bcc6-ae60208bd417 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:27.457Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03144841-47cf-4105-bcc6-ae60208bd417 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27db2cc8 2021-05-28T00:21:27.457Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03144841-47cf-4105-bcc6-ae60208bd417 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:27.457Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03144841-47cf-4105-bcc6-ae60208bd417 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:28.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=371288c2-743b-45be-9ecf-64fe53396715 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:28.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=371288c2-743b-45be-9ecf-64fe53396715 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23e8c1f2 2021-05-28T00:21:28.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=371288c2-743b-45be-9ecf-64fe53396715 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:28.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=371288c2-743b-45be-9ecf-64fe53396715 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:29.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:29.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22466617 2021-05-28T00:21:29.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:29.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:32.386Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:21:32.386Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:21:32.386Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:21:32.386Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b9d9ce75-0145-4aa2-908b-ac29b6252c2e ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:21:33.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e90a9b3-2765-425b-8016-d4bbc7f75e53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:33.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e90a9b3-2765-425b-8016-d4bbc7f75e53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2566664a 2021-05-28T00:21:33.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e90a9b3-2765-425b-8016-d4bbc7f75e53 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:33.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e90a9b3-2765-425b-8016-d4bbc7f75e53 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:34.473Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=1e90a9b3-2765-425b-8016-d4bbc7f75e53 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:21:34.473Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=1e90a9b3-2765-425b-8016-d4bbc7f75e53 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:21:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:21:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:21:35.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd904fc3-7220-45d0-9dff-37d4b3eef139 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:35.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd904fc3-7220-45d0-9dff-37d4b3eef139 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e270e95 2021-05-28T00:21:35.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd904fc3-7220-45d0-9dff-37d4b3eef139 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:35.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd904fc3-7220-45d0-9dff-37d4b3eef139 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:21:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:21:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:21:37.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=064e4525-5607-4aa0-a249-9b1fc3f1101f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:37.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=064e4525-5607-4aa0-a249-9b1fc3f1101f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f40b198 2021-05-28T00:21:37.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=064e4525-5607-4aa0-a249-9b1fc3f1101f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:37.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=064e4525-5607-4aa0-a249-9b1fc3f1101f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:40.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=553fda4e-bbe3-4305-9d2f-9b1b9373e15f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:40.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=553fda4e-bbe3-4305-9d2f-9b1b9373e15f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3982c60c 2021-05-28T00:21:40.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=553fda4e-bbe3-4305-9d2f-9b1b9373e15f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:40.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=553fda4e-bbe3-4305-9d2f-9b1b9373e15f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:44.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff226b3c-6a40-43ca-97e1-d8e1cddfaa7e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:44.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff226b3c-6a40-43ca-97e1-d8e1cddfaa7e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fcbcdcd 2021-05-28T00:21:44.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff226b3c-6a40-43ca-97e1-d8e1cddfaa7e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:44.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff226b3c-6a40-43ca-97e1-d8e1cddfaa7e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:46.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cd3ffe5-05d7-4272-98cf-05e182362431 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:46.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cd3ffe5-05d7-4272-98cf-05e182362431 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@491e2107 2021-05-28T00:21:46.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cd3ffe5-05d7-4272-98cf-05e182362431 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:46.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cd3ffe5-05d7-4272-98cf-05e182362431 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:49.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=879a8a00-d6b0-43eb-94ef-c5bec783f3bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:49.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=879a8a00-d6b0-43eb-94ef-c5bec783f3bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@637895cd 2021-05-28T00:21:49.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=879a8a00-d6b0-43eb-94ef-c5bec783f3bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:49.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=879a8a00-d6b0-43eb-94ef-c5bec783f3bc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:21:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:21:52.128Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:21:52.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d63a987c-55ed-4940-a1cb-cb4703f6853d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:52.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d63a987c-55ed-4940-a1cb-cb4703f6853d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ce59d08 2021-05-28T00:21:52.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d63a987c-55ed-4940-a1cb-cb4703f6853d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:52.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d63a987c-55ed-4940-a1cb-cb4703f6853d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:56.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5701b42a-3c56-4e68-bdbc-9c009668ddb5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:56.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5701b42a-3c56-4e68-bdbc-9c009668ddb5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5dd184fc 2021-05-28T00:21:56.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5701b42a-3c56-4e68-bdbc-9c009668ddb5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:56.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5701b42a-3c56-4e68-bdbc-9c009668ddb5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:21:58.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c035b60a-401f-4865-8cbc-ea6442ba055b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:21:58.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c035b60a-401f-4865-8cbc-ea6442ba055b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@657f8058 2021-05-28T00:21:58.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c035b60a-401f-4865-8cbc-ea6442ba055b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:21:58.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c035b60a-401f-4865-8cbc-ea6442ba055b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:01.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbf1705-0fa2-40dc-a885-4bf4e3531331 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:01.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbf1705-0fa2-40dc-a885-4bf4e3531331 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@428eb7c7 2021-05-28T00:22:01.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbf1705-0fa2-40dc-a885-4bf4e3531331 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:01.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dbf1705-0fa2-40dc-a885-4bf4e3531331 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:05.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fd9d2fc-13e9-4ccc-8147-6a100d089085 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:05.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fd9d2fc-13e9-4ccc-8147-6a100d089085 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22637f43 2021-05-28T00:22:05.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fd9d2fc-13e9-4ccc-8147-6a100d089085 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:05.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fd9d2fc-13e9-4ccc-8147-6a100d089085 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:22:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:22:08.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb7aad2a-f771-4264-912a-7ec36e6613d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:08.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb7aad2a-f771-4264-912a-7ec36e6613d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7344e997 2021-05-28T00:22:08.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb7aad2a-f771-4264-912a-7ec36e6613d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:08.484Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb7aad2a-f771-4264-912a-7ec36e6613d3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:10.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a54fa4da-e5c1-429e-b92a-26f0d43bd8ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:10.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a54fa4da-e5c1-429e-b92a-26f0d43bd8ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a20cae7 2021-05-28T00:22:10.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a54fa4da-e5c1-429e-b92a-26f0d43bd8ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:10.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a54fa4da-e5c1-429e-b92a-26f0d43bd8ca PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:13.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a054f8-63a4-4d8b-808a-5d186f17d149 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:13.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a054f8-63a4-4d8b-808a-5d186f17d149 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5da768eb 2021-05-28T00:22:13.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a054f8-63a4-4d8b-808a-5d186f17d149 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:13.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90a054f8-63a4-4d8b-808a-5d186f17d149 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:16.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11973381-3a9e-4341-9029-3d55e4e5a479 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:16.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11973381-3a9e-4341-9029-3d55e4e5a479 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64079d7b 2021-05-28T00:22:16.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11973381-3a9e-4341-9029-3d55e4e5a479 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:16.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11973381-3a9e-4341-9029-3d55e4e5a479 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:20.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fa20dc5-c643-42ab-b52e-640a9eed5428 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:20.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fa20dc5-c643-42ab-b52e-640a9eed5428 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30dbc83d 2021-05-28T00:22:20.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fa20dc5-c643-42ab-b52e-640a9eed5428 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:20.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fa20dc5-c643-42ab-b52e-640a9eed5428 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:22:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:22:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:22:23.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:22:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8b5ce9-a0c8-4e2e-a219-33e8baa67622 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8b5ce9-a0c8-4e2e-a219-33e8baa67622 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c405b33 2021-05-28T00:22:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8b5ce9-a0c8-4e2e-a219-33e8baa67622 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8b5ce9-a0c8-4e2e-a219-33e8baa67622 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:25.780Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:22:27.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:27.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ac68060 2021-05-28T00:22:27.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:27.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:30.388Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:22:30.388Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:22:30.388Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:22:30.389Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=44fac399-acbd-4b73-9a97-1bd3acbb4766 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:22:34.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1578eb5-1f2f-4fa2-ac21-ae8a96069a28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:34.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1578eb5-1f2f-4fa2-ac21-ae8a96069a28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62722e06 2021-05-28T00:22:34.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1578eb5-1f2f-4fa2-ac21-ae8a96069a28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:34.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1578eb5-1f2f-4fa2-ac21-ae8a96069a28 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:35.469Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e1578eb5-1f2f-4fa2-ac21-ae8a96069a28 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:22:35.469Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e1578eb5-1f2f-4fa2-ac21-ae8a96069a28 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:22:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:22:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:22:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:22:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:22:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:22:38.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8930e7d4-880f-40a3-acca-2cb203804758 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:38.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8930e7d4-880f-40a3-acca-2cb203804758 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5241c335 2021-05-28T00:22:38.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8930e7d4-880f-40a3-acca-2cb203804758 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:38.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8930e7d4-880f-40a3-acca-2cb203804758 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:40.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02ad39f5-c867-4cc6-8768-df7c5085241b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:40.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02ad39f5-c867-4cc6-8768-df7c5085241b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c8a8c9d 2021-05-28T00:22:40.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02ad39f5-c867-4cc6-8768-df7c5085241b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:40.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02ad39f5-c867-4cc6-8768-df7c5085241b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:41.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913df86-21cb-42e0-bdbf-ee2db89b9f2d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:41.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913df86-21cb-42e0-bdbf-ee2db89b9f2d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f7b9b14 2021-05-28T00:22:41.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913df86-21cb-42e0-bdbf-ee2db89b9f2d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:41.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913df86-21cb-42e0-bdbf-ee2db89b9f2d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:43.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5763c4ac-6396-4027-ba25-8ef378dbc63a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:43.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5763c4ac-6396-4027-ba25-8ef378dbc63a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1707fa9e 2021-05-28T00:22:43.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5763c4ac-6396-4027-ba25-8ef378dbc63a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:43.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5763c4ac-6396-4027-ba25-8ef378dbc63a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:45.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f87a4e-90f3-4f4e-aa33-252dcd7bfccf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:45.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f87a4e-90f3-4f4e-aa33-252dcd7bfccf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b03db20 2021-05-28T00:22:45.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f87a4e-90f3-4f4e-aa33-252dcd7bfccf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:45.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2f87a4e-90f3-4f4e-aa33-252dcd7bfccf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:47.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b72fbc1-a9b4-4f96-aadf-14960b62c0a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:47.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b72fbc1-a9b4-4f96-aadf-14960b62c0a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43e548b9 2021-05-28T00:22:47.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b72fbc1-a9b4-4f96-aadf-14960b62c0a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:47.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b72fbc1-a9b4-4f96-aadf-14960b62c0a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d87120e-1901-4a17-9da0-70b39fbc24bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d87120e-1901-4a17-9da0-70b39fbc24bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3035c8ea 2021-05-28T00:22:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d87120e-1901-4a17-9da0-70b39fbc24bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d87120e-1901-4a17-9da0-70b39fbc24bf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:22:50.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:22:52.118Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:22:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06abaf5a-292b-41d1-a47b-c46238a79d2a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06abaf5a-292b-41d1-a47b-c46238a79d2a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@184e0778 2021-05-28T00:22:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06abaf5a-292b-41d1-a47b-c46238a79d2a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06abaf5a-292b-41d1-a47b-c46238a79d2a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:56.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:22:56.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d41058a 2021-05-28T00:22:56.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:22:56.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:22:59.771Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:22:59.772Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:22:59.772Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:22:59.772Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e6583cce-2d21-4f08-a4ca-f2e3ebd8d2ba ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:23:04.781Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:04.856Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:05.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:23:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:23:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:10.787Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:23:10.788Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:10.860Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:23:10.861Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:11.471Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:14.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:17.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:17.867Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:23:17.868Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:17.871Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:23:17.872Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:19.822Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:19.823Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:19.823Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24083c1b 2021-05-28T00:23:19.823Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:19.823Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:23:20.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:23:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:23:21.877Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:23:21.878Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 8000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:21.957Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:23:21.958Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 8000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:22.725Z [cluster1-reconnection-0] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 1000 milliseconds 2021-05-28T00:23:22.863Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:23:22.864Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:23:22.864Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:22.864Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=c9589422-1fcf-4004-aa93-dace3ea43111 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:23:22.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:22.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5279671b 2021-05-28T00:23:22.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:22.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:22.870Z [cluster2-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:23:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:23.726Z [cluster1-reconnection-0] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 2000 milliseconds 2021-05-28T00:23:25.726Z [cluster1-reconnection-0] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 4000 milliseconds 2021-05-28T00:23:25.899Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:23:25.899Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:23:25.900Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:25.900Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=a3052a8c-b98e-4bcc-8a30-0559647954ca ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:23:25.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac739bfb-6790-415e-ae95-f656eb377dc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:25.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac739bfb-6790-415e-ae95-f656eb377dc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@725c4b8e 2021-05-28T00:23:25.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac739bfb-6790-415e-ae95-f656eb377dc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:25.900Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac739bfb-6790-415e-ae95-f656eb377dc1 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:28.928Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ac739bfb-6790-415e-ae95-f656eb377dc1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:23:28.928Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ac739bfb-6790-415e-ae95-f656eb377dc1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:23:28.929Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a775da1-ce7a-47c5-9979-36bc17a4d5e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:28.929Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a775da1-ce7a-47c5-9979-36bc17a4d5e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@776bd7cd 2021-05-28T00:23:28.929Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a775da1-ce7a-47c5-9979-36bc17a4d5e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:28.929Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a775da1-ce7a-47c5-9979-36bc17a4d5e4 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:29.727Z [cluster1-reconnection-0] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 8000 milliseconds 2021-05-28T00:23:29.975Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:23:29.976Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.76.215:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:29.982Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:23:29.983Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:23:31.971Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0a775da1-ce7a-47c5-9979-36bc17a4d5e4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:23:31.971Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0a775da1-ce7a-47c5-9979-36bc17a4d5e4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:23:31.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9ce8471-2adc-47dd-8735-ac195cbe8da7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:31.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9ce8471-2adc-47dd-8735-ac195cbe8da7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a1b9f52 2021-05-28T00:23:31.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9ce8471-2adc-47dd-8735-ac195cbe8da7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:31.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9ce8471-2adc-47dd-8735-ac195cbe8da7 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:35.000Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c9ce8471-2adc-47dd-8735-ac195cbe8da7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:23:35.001Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c9ce8471-2adc-47dd-8735-ac195cbe8da7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:23:35.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e255e33-c546-4de9-ab05-8a385543355d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:35.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e255e33-c546-4de9-ab05-8a385543355d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@248621bf 2021-05-28T00:23:35.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e255e33-c546-4de9-ab05-8a385543355d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:35.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e255e33-c546-4de9-ab05-8a385543355d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:23:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:23:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:23:36.339Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e85ef6-1c6a-4982-9d8d-5ff7cd533062 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:36.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e85ef6-1c6a-4982-9d8d-5ff7cd533062 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35b01633 2021-05-28T00:23:36.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e85ef6-1c6a-4982-9d8d-5ff7cd533062 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:36.340Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70e85ef6-1c6a-4982-9d8d-5ff7cd533062 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:23:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:23:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:23:37.480Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:23:37.727Z [cluster1-reconnection-0] ERROR c.d.driver.core.ControlConnection [Control connection] Cannot connect to any host, scheduling retry in 16000 milliseconds 2021-05-28T00:23:38.000Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33afc690-0b62-4214-b85e-9673cac12561 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:38.000Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33afc690-0b62-4214-b85e-9673cac12561 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a4673bd 2021-05-28T00:23:38.000Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33afc690-0b62-4214-b85e-9673cac12561 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:38.000Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33afc690-0b62-4214-b85e-9673cac12561 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:38.185Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=33afc690-0b62-4214-b85e-9673cac12561 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:38.185Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=33afc690-0b62-4214-b85e-9673cac12561 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:23:38.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1622167c-0c2d-4f2a-aec7-8bd46311653b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:38.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1622167c-0c2d-4f2a-aec7-8bd46311653b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fd82c93 2021-05-28T00:23:38.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1622167c-0c2d-4f2a-aec7-8bd46311653b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:38.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1622167c-0c2d-4f2a-aec7-8bd46311653b PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:38.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e469639c-03c1-4031-b5da-2cbaacf3bd90 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:38.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e469639c-03c1-4031-b5da-2cbaacf3bd90 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53537e02 2021-05-28T00:23:38.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e469639c-03c1-4031-b5da-2cbaacf3bd90 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:38.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e469639c-03c1-4031-b5da-2cbaacf3bd90 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:38.490Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28df7420-a84d-492f-85fc-d24f55602e2d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:38.490Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28df7420-a84d-492f-85fc-d24f55602e2d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6667ebd5 2021-05-28T00:23:38.490Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28df7420-a84d-492f-85fc-d24f55602e2d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:38.501Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28df7420-a84d-492f-85fc-d24f55602e2d PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:38.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=526cd6a5-dd12-48fa-b0f4-c9de368393a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:38.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=526cd6a5-dd12-48fa-b0f4-c9de368393a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3fe5b73f 2021-05-28T00:23:38.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=526cd6a5-dd12-48fa-b0f4-c9de368393a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:38.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=526cd6a5-dd12-48fa-b0f4-c9de368393a1 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:39.581Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=526cd6a5-dd12-48fa-b0f4-c9de368393a1 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:39.581Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=526cd6a5-dd12-48fa-b0f4-c9de368393a1 ErrorCategory=ERROR ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:23:39.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0285f9d-8991-46d4-94a2-2d88f95bc3ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:39.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0285f9d-8991-46d4-94a2-2d88f95bc3ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e7a8e4 2021-05-28T00:23:39.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0285f9d-8991-46d4-94a2-2d88f95bc3ce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:39.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0285f9d-8991-46d4-94a2-2d88f95bc3ce PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:39.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca9484e4-49d8-4f5e-8669-46e399bb4103 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:39.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca9484e4-49d8-4f5e-8669-46e399bb4103 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@490c65b4 2021-05-28T00:23:39.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca9484e4-49d8-4f5e-8669-46e399bb4103 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:39.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca9484e4-49d8-4f5e-8669-46e399bb4103 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:40.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cdb4c85-2f37-4844-995b-f8bf68542053 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:40.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cdb4c85-2f37-4844-995b-f8bf68542053 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10ec1603 2021-05-28T00:23:40.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cdb4c85-2f37-4844-995b-f8bf68542053 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:40.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cdb4c85-2f37-4844-995b-f8bf68542053 PartnerName=SDC.BE The cassandra down nodes number is 3 2021-05-28T00:23:43.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0754c3-c345-48f4-9e0e-6792f2eb06ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:43.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0754c3-c345-48f4-9e0e-6792f2eb06ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@222176e 2021-05-28T00:23:43.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0754c3-c345-48f4-9e0e-6792f2eb06ff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:43.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0754c3-c345-48f4-9e0e-6792f2eb06ff PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:23:46.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fffa8785-dc6b-4664-9dfa-b4de68f04f80 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:46.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fffa8785-dc6b-4664-9dfa-b4de68f04f80 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ebe3f46 2021-05-28T00:23:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fffa8785-dc6b-4664-9dfa-b4de68f04f80 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fffa8785-dc6b-4664-9dfa-b4de68f04f80 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:23:46.768Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fffa8785-dc6b-4664-9dfa-b4de68f04f80 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:46.769Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fffa8785-dc6b-4664-9dfa-b4de68f04f80 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72db5ff2-1ecd-4026-809a-ffd810955087 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72db5ff2-1ecd-4026-809a-ffd810955087 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14b5dce0 2021-05-28T00:23:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72db5ff2-1ecd-4026-809a-ffd810955087 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72db5ff2-1ecd-4026-809a-ffd810955087 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:23:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:23:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:23:52.122Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:23:52.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:52.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64198c7b 2021-05-28T00:23:52.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:52.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:23:52.770Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e PartnerName=SDC.BE I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {s}->https://sdc-onboarding-be.onap:8445: The target server failed to respond 2021-05-28T00:23:52.770Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e PartnerName=SDC.BE Retrying request to {s}->https://sdc-onboarding-be.onap:8445 2021-05-28T00:23:52.773Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:23:52.774Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:23:52.774Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:52.774Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a61b74ed-60f8-45cc-88ab-775942fcbb5e ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:23:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7153b0a6-cb3c-432b-a6d1-19f83cc61f91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7153b0a6-cb3c-432b-a6d1-19f83cc61f91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39193b6a 2021-05-28T00:23:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7153b0a6-cb3c-432b-a6d1-19f83cc61f91 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7153b0a6-cb3c-432b-a6d1-19f83cc61f91 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:23:55.882Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7153b0a6-cb3c-432b-a6d1-19f83cc61f91 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:23:55.883Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7153b0a6-cb3c-432b-a6d1-19f83cc61f91 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:23:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=824cefb4-3a06-4813-991d-27bc8d0a98f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:23:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=824cefb4-3a06-4813-991d-27bc8d0a98f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ba38c24 2021-05-28T00:23:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=824cefb4-3a06-4813-991d-27bc8d0a98f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:23:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=824cefb4-3a06-4813-991d-27bc8d0a98f7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:23:58.770Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=824cefb4-3a06-4813-991d-27bc8d0a98f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:23:58.771Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=824cefb4-3a06-4813-991d-27bc8d0a98f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:01.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac1be8ea-a4a2-4520-9118-e14e5840c602 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:01.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac1be8ea-a4a2-4520-9118-e14e5840c602 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e112a82 2021-05-28T00:24:01.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac1be8ea-a4a2-4520-9118-e14e5840c602 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:01.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac1be8ea-a4a2-4520-9118-e14e5840c602 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:01.664Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ac1be8ea-a4a2-4520-9118-e14e5840c602 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:24:01.665Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ac1be8ea-a4a2-4520-9118-e14e5840c602 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3886e2-2c56-4934-9ffe-0be9158bdafc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3886e2-2c56-4934-9ffe-0be9158bdafc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7083da8 2021-05-28T00:24:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3886e2-2c56-4934-9ffe-0be9158bdafc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3886e2-2c56-4934-9ffe-0be9158bdafc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:04.880Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7b3886e2-2c56-4934-9ffe-0be9158bdafc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:24:04.880Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7b3886e2-2c56-4934-9ffe-0be9158bdafc ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:24:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:24:07.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30adb22e-0079-475f-ba35-3eeb4dfd88f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:07.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30adb22e-0079-475f-ba35-3eeb4dfd88f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62d90bb1 2021-05-28T00:24:07.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30adb22e-0079-475f-ba35-3eeb4dfd88f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:07.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30adb22e-0079-475f-ba35-3eeb4dfd88f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:07.475Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=30adb22e-0079-475f-ba35-3eeb4dfd88f1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:24:07.476Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=30adb22e-0079-475f-ba35-3eeb4dfd88f1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b0ba47b-4762-4eab-ae9a-65f19bdbbaa2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b0ba47b-4762-4eab-ae9a-65f19bdbbaa2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ef47127 2021-05-28T00:24:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b0ba47b-4762-4eab-ae9a-65f19bdbbaa2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b0ba47b-4762-4eab-ae9a-65f19bdbbaa2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:10.872Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3b0ba47b-4762-4eab-ae9a-65f19bdbbaa2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:24:10.872Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3b0ba47b-4762-4eab-ae9a-65f19bdbbaa2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:13.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d12d2d56-67fd-4d71-8963-29a7778fe001 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:13.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d12d2d56-67fd-4d71-8963-29a7778fe001 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a8ab77d 2021-05-28T00:24:13.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d12d2d56-67fd-4d71-8963-29a7778fe001 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:13.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d12d2d56-67fd-4d71-8963-29a7778fe001 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:13.673Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d12d2d56-67fd-4d71-8963-29a7778fe001 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:24:13.673Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d12d2d56-67fd-4d71-8963-29a7778fe001 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:16.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6870bc5c-1fe5-4393-ae68-2a080c12ff5a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:16.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6870bc5c-1fe5-4393-ae68-2a080c12ff5a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8d9db1 2021-05-28T00:24:16.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6870bc5c-1fe5-4393-ae68-2a080c12ff5a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:16.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6870bc5c-1fe5-4393-ae68-2a080c12ff5a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:16.466Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6870bc5c-1fe5-4393-ae68-2a080c12ff5a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:24:16.467Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6870bc5c-1fe5-4393-ae68-2a080c12ff5a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:19.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=123d8518-82ad-4902-b35c-b9626563f279 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:19.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=123d8518-82ad-4902-b35c-b9626563f279 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ab0499e 2021-05-28T00:24:19.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=123d8518-82ad-4902-b35c-b9626563f279 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:19.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=123d8518-82ad-4902-b35c-b9626563f279 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:24:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:24:22.883Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=123d8518-82ad-4902-b35c-b9626563f279 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:24:22.883Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=123d8518-82ad-4902-b35c-b9626563f279 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:23.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4e143d-09c2-442a-ab24-cd1f743b72b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:23.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4e143d-09c2-442a-ab24-cd1f743b72b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63276b3 2021-05-28T00:24:23.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4e143d-09c2-442a-ab24-cd1f743b72b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:23.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4e143d-09c2-442a-ab24-cd1f743b72b8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:26.775Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ff4e143d-09c2-442a-ab24-cd1f743b72b8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:24:26.775Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ff4e143d-09c2-442a-ab24-cd1f743b72b8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:27.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3878f8ef-518a-411e-859d-6946b0d0453a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:27.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3878f8ef-518a-411e-859d-6946b0d0453a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a5414d4 2021-05-28T00:24:27.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3878f8ef-518a-411e-859d-6946b0d0453a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:27.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3878f8ef-518a-411e-859d-6946b0d0453a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:30.275Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3878f8ef-518a-411e-859d-6946b0d0453a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:24:30.275Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3878f8ef-518a-411e-859d-6946b0d0453a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:31.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d361730-2fe1-49f3-86ff-abdaa202de62 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:31.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d361730-2fe1-49f3-86ff-abdaa202de62 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7590ac0d 2021-05-28T00:24:31.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d361730-2fe1-49f3-86ff-abdaa202de62 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:31.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d361730-2fe1-49f3-86ff-abdaa202de62 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:34.086Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3d361730-2fe1-49f3-86ff-abdaa202de62 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:24:34.086Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3d361730-2fe1-49f3-86ff-abdaa202de62 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:34.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5eecd5d-d943-4195-ace4-a44459d9d4a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:34.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5eecd5d-d943-4195-ace4-a44459d9d4a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fc51e72 2021-05-28T00:24:34.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5eecd5d-d943-4195-ace4-a44459d9d4a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:34.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5eecd5d-d943-4195-ace4-a44459d9d4a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:24:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:24:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:24:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:24:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:24:37.981Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d5eecd5d-d943-4195-ace4-a44459d9d4a5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:24:37.981Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d5eecd5d-d943-4195-ace4-a44459d9d4a5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:38.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afa49c99-95e5-4628-87c3-c2b94498991f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:38.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afa49c99-95e5-4628-87c3-c2b94498991f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@97ba5f5 2021-05-28T00:24:38.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afa49c99-95e5-4628-87c3-c2b94498991f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:38.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afa49c99-95e5-4628-87c3-c2b94498991f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:41.782Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=afa49c99-95e5-4628-87c3-c2b94498991f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:24:41.783Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=afa49c99-95e5-4628-87c3-c2b94498991f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:42.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a267dacd-10c6-4e3e-8175-293b2b86e7e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:42.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a267dacd-10c6-4e3e-8175-293b2b86e7e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11ad4864 2021-05-28T00:24:42.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a267dacd-10c6-4e3e-8175-293b2b86e7e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:42.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a267dacd-10c6-4e3e-8175-293b2b86e7e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:45.680Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a267dacd-10c6-4e3e-8175-293b2b86e7e3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:24:45.680Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a267dacd-10c6-4e3e-8175-293b2b86e7e3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:24:46.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0efb7ed-ff25-42f7-ab0a-2341f8c5f210 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:46.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0efb7ed-ff25-42f7-ab0a-2341f8c5f210 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6083882c 2021-05-28T00:24:46.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0efb7ed-ff25-42f7-ab0a-2341f8c5f210 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:46.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0efb7ed-ff25-42f7-ab0a-2341f8c5f210 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:24:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:24:52.172Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:24:52.254Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e0efb7ed-ff25-42f7-ab0a-2341f8c5f210 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:24:52.254Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e0efb7ed-ff25-42f7-ab0a-2341f8c5f210 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:24:53.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e2a41d-bbc6-4a6a-96eb-a0d2cbeb185e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:53.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e2a41d-bbc6-4a6a-96eb-a0d2cbeb185e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a49144a 2021-05-28T00:24:53.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e2a41d-bbc6-4a6a-96eb-a0d2cbeb185e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:53.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e2a41d-bbc6-4a6a-96eb-a0d2cbeb185e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:53.642Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a0e2a41d-bbc6-4a6a-96eb-a0d2cbeb185e ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:24:53.642Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a0e2a41d-bbc6-4a6a-96eb-a0d2cbeb185e ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:24:54.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97fcd59b-0d37-4c3b-b588-44d282cac03c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:54.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97fcd59b-0d37-4c3b-b588-44d282cac03c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c64756f 2021-05-28T00:24:54.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97fcd59b-0d37-4c3b-b588-44d282cac03c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:54.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97fcd59b-0d37-4c3b-b588-44d282cac03c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:55.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b167c3-00f6-4454-9a12-2189f3f437a2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:55.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b167c3-00f6-4454-9a12-2189f3f437a2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49cef5f2 2021-05-28T00:24:55.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b167c3-00f6-4454-9a12-2189f3f437a2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:55.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b167c3-00f6-4454-9a12-2189f3f437a2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:56.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b3e1ab-5cbe-4720-b2e3-6a15fb53090d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:56.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b3e1ab-5cbe-4720-b2e3-6a15fb53090d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ba743c5 2021-05-28T00:24:56.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b3e1ab-5cbe-4720-b2e3-6a15fb53090d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:56.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b3e1ab-5cbe-4720-b2e3-6a15fb53090d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:24:58.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9afefe3-c029-4e70-9fd1-11926e24e321 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:24:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9afefe3-c029-4e70-9fd1-11926e24e321 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79d2de42 2021-05-28T00:24:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9afefe3-c029-4e70-9fd1-11926e24e321 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:24:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9afefe3-c029-4e70-9fd1-11926e24e321 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:00.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77368e2-168c-4098-9841-f659854b81d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:00.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77368e2-168c-4098-9841-f659854b81d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46a2031a 2021-05-28T00:25:00.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77368e2-168c-4098-9841-f659854b81d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:00.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77368e2-168c-4098-9841-f659854b81d4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:02.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4f6c40e-8191-4632-a855-01c01cd4da5f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:02.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4f6c40e-8191-4632-a855-01c01cd4da5f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@787d0f59 2021-05-28T00:25:02.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4f6c40e-8191-4632-a855-01c01cd4da5f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:02.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4f6c40e-8191-4632-a855-01c01cd4da5f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:04.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bea5fcc-37d9-4d96-be32-b04b7e574d33 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:04.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bea5fcc-37d9-4d96-be32-b04b7e574d33 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9b5c2f8 2021-05-28T00:25:04.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bea5fcc-37d9-4d96-be32-b04b7e574d33 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:04.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bea5fcc-37d9-4d96-be32-b04b7e574d33 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:25:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:25:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64b4936-3554-4565-aaa9-95aa292d5204 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64b4936-3554-4565-aaa9-95aa292d5204 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66fa3d42 2021-05-28T00:25:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64b4936-3554-4565-aaa9-95aa292d5204 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c64b4936-3554-4565-aaa9-95aa292d5204 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:10.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29df3b02-4bd2-41ca-be03-b8069e2fb465 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:10.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29df3b02-4bd2-41ca-be03-b8069e2fb465 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30164854 2021-05-28T00:25:10.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29df3b02-4bd2-41ca-be03-b8069e2fb465 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:10.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29df3b02-4bd2-41ca-be03-b8069e2fb465 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:14.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4232079-4773-45b8-a020-3d33b434a1e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:14.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4232079-4773-45b8-a020-3d33b434a1e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20019ec8 2021-05-28T00:25:14.383Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4232079-4773-45b8-a020-3d33b434a1e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:14.383Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4232079-4773-45b8-a020-3d33b434a1e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:17.007Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d66ac2b3-e607-4275-9c29-56825b522566 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:17.007Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d66ac2b3-e607-4275-9c29-56825b522566 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7842a32 2021-05-28T00:25:17.007Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d66ac2b3-e607-4275-9c29-56825b522566 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:17.007Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d66ac2b3-e607-4275-9c29-56825b522566 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:19.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0efe6e-fdac-4964-8e94-63dc0cb46a7f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:19.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0efe6e-fdac-4964-8e94-63dc0cb46a7f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30a307ad 2021-05-28T00:25:19.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0efe6e-fdac-4964-8e94-63dc0cb46a7f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:19.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0efe6e-fdac-4964-8e94-63dc0cb46a7f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:25:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:25:22.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1fff232-365a-45ff-a533-9fddfe67bc26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:22.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1fff232-365a-45ff-a533-9fddfe67bc26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32f5ad05 2021-05-28T00:25:22.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1fff232-365a-45ff-a533-9fddfe67bc26 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:22.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1fff232-365a-45ff-a533-9fddfe67bc26 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:25.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc07f4ff-03b6-4f18-b03d-31f480b5674d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:25.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc07f4ff-03b6-4f18-b03d-31f480b5674d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1eae924d 2021-05-28T00:25:25.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc07f4ff-03b6-4f18-b03d-31f480b5674d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:25.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc07f4ff-03b6-4f18-b03d-31f480b5674d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15b55681-e589-428f-a9e8-0f846c4ae3b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15b55681-e589-428f-a9e8-0f846c4ae3b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38ddfe10 2021-05-28T00:25:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15b55681-e589-428f-a9e8-0f846c4ae3b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15b55681-e589-428f-a9e8-0f846c4ae3b4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:32.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bb9119b-9af7-4c41-9537-7bc917781ab6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:32.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bb9119b-9af7-4c41-9537-7bc917781ab6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10eb18c9 2021-05-28T00:25:32.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bb9119b-9af7-4c41-9537-7bc917781ab6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:32.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bb9119b-9af7-4c41-9537-7bc917781ab6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:35.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f7a402-62bb-40ea-b9da-e489e3768c28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:35.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f7a402-62bb-40ea-b9da-e489e3768c28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f690254 2021-05-28T00:25:35.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f7a402-62bb-40ea-b9da-e489e3768c28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:35.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f7a402-62bb-40ea-b9da-e489e3768c28 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:25:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:25:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:25:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:25:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:25:37.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e48d348-1a7a-4659-82c5-4e424c82e789 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:37.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e48d348-1a7a-4659-82c5-4e424c82e789 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@653fc50e 2021-05-28T00:25:37.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e48d348-1a7a-4659-82c5-4e424c82e789 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:37.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e48d348-1a7a-4659-82c5-4e424c82e789 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:41.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18359b61-1f49-4a7f-b126-16fb13459cde PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:41.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18359b61-1f49-4a7f-b126-16fb13459cde PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32459ac7 2021-05-28T00:25:41.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18359b61-1f49-4a7f-b126-16fb13459cde PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:41.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18359b61-1f49-4a7f-b126-16fb13459cde PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:43.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79dd00e-e171-439b-9de3-603c1e275efc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:43.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79dd00e-e171-439b-9de3-603c1e275efc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a20e116 2021-05-28T00:25:43.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79dd00e-e171-439b-9de3-603c1e275efc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:43.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79dd00e-e171-439b-9de3-603c1e275efc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:46.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c3e90f8-1ebf-454f-9d19-e04d5e521dd7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:46.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c3e90f8-1ebf-454f-9d19-e04d5e521dd7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d1c37a4 2021-05-28T00:25:46.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c3e90f8-1ebf-454f-9d19-e04d5e521dd7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:46.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c3e90f8-1ebf-454f-9d19-e04d5e521dd7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:49.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de30df5f-17c5-43e0-830e-eba9f9c613ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:49.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de30df5f-17c5-43e0-830e-eba9f9c613ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@269092bc 2021-05-28T00:25:49.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de30df5f-17c5-43e0-830e-eba9f9c613ff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:49.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de30df5f-17c5-43e0-830e-eba9f9c613ff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:25:50.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:25:52.264Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:25:53.397Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75a8ffb4-4fcc-4096-a545-05e16d42a1d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:53.397Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75a8ffb4-4fcc-4096-a545-05e16d42a1d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@381db70b 2021-05-28T00:25:53.397Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75a8ffb4-4fcc-4096-a545-05e16d42a1d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:53.398Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75a8ffb4-4fcc-4096-a545-05e16d42a1d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:56.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b3b2d10-3f10-42a8-8e8f-056afd5ff588 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:56.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b3b2d10-3f10-42a8-8e8f-056afd5ff588 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4548d94f 2021-05-28T00:25:56.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b3b2d10-3f10-42a8-8e8f-056afd5ff588 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:56.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b3b2d10-3f10-42a8-8e8f-056afd5ff588 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:25:59.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e76aa2e-aac6-49cb-8169-aac84fd5d3d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:25:59.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e76aa2e-aac6-49cb-8169-aac84fd5d3d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19357755 2021-05-28T00:25:59.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e76aa2e-aac6-49cb-8169-aac84fd5d3d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:25:59.791Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e76aa2e-aac6-49cb-8169-aac84fd5d3d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:02.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f7beb4-3685-4faf-88de-9ef825b72ef3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:02.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f7beb4-3685-4faf-88de-9ef825b72ef3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b1203bd 2021-05-28T00:26:02.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f7beb4-3685-4faf-88de-9ef825b72ef3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:02.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f7beb4-3685-4faf-88de-9ef825b72ef3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:05.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:05.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ee24e3b 2021-05-28T00:26:05.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:05.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:05.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:26:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:26:08.080Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:26:08.081Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:26:08.081Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:08.081Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=aa10815e-8771-4bb2-b36a-90964f8f2017 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:11.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cfdd27-78c3-4015-9fa0-ae433a2ae69d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:11.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cfdd27-78c3-4015-9fa0-ae433a2ae69d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6de36009 2021-05-28T00:26:11.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cfdd27-78c3-4015-9fa0-ae433a2ae69d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:11.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cfdd27-78c3-4015-9fa0-ae433a2ae69d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:12.164Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b9cfdd27-78c3-4015-9fa0-ae433a2ae69d ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:12.164Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b9cfdd27-78c3-4015-9fa0-ae433a2ae69d ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:12.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1d15ca7-4fec-42b8-afe0-2af2a5f21b45 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:12.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1d15ca7-4fec-42b8-afe0-2af2a5f21b45 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e800383 2021-05-28T00:26:12.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1d15ca7-4fec-42b8-afe0-2af2a5f21b45 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:12.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1d15ca7-4fec-42b8-afe0-2af2a5f21b45 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:14.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3035213-0e7c-473c-8283-c678be2cd293 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:14.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3035213-0e7c-473c-8283-c678be2cd293 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50996eb9 2021-05-28T00:26:14.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3035213-0e7c-473c-8283-c678be2cd293 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:14.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3035213-0e7c-473c-8283-c678be2cd293 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:16.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=521ec3f6-633e-4ba0-9c8b-7b81b193709d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:16.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=521ec3f6-633e-4ba0-9c8b-7b81b193709d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72d5486a 2021-05-28T00:26:16.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=521ec3f6-633e-4ba0-9c8b-7b81b193709d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:16.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=521ec3f6-633e-4ba0-9c8b-7b81b193709d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:26:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:26:21.477Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a0d99fda-d281-40af-ac17-a0901527aa14 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:21.477016Z InvocationID=ff09a5e5-7395-45b3-bb28-b9cfaf716747 LogTimestamp=2021-05-28T00:26:21.477157Z Entering 2021-05-28T00:26:21.478Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a0d99fda-d281-40af-ac17-a0901527aa14 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:21.477016Z InvocationID=ff09a5e5-7395-45b3-bb28-b9cfaf716747 LogTimestamp=2021-05-28T00:26:21.477157Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:26:21.497Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a88599b-219f-4415-9d28-440e91564d8c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:21.497Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a88599b-219f-4415-9d28-440e91564d8c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54ab57d9 2021-05-28T00:26:21.497Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a88599b-219f-4415-9d28-440e91564d8c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:21.497Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a88599b-219f-4415-9d28-440e91564d8c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:21.769Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=291 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:26:21.478Z InvocationID=3737b007-dd05-4945-bc67-62ef7b097314 LogTimestamp=2021-05-28T00:26:21.769717Z ResponseDesc=OK Exiting. 2021-05-28T00:26:21.770Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:26:21.877Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=195203 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:23:06.674Z InvocationID=5490bf3c-0da9-4958-80e7-43babef5cb4b ErrorCode=500 LogTimestamp=2021-05-28T00:26:21.877343Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:21.878Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:26:21.970Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bea28a33-9642-4363-a352-605e85c31aef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:21.970342Z InvocationID=3795564a-f098-4850-af3d-9706022fc01b LogTimestamp=2021-05-28T00:26:21.970400Z Entering 2021-05-28T00:26:21.970Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bea28a33-9642-4363-a352-605e85c31aef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:21.970342Z InvocationID=3795564a-f098-4850-af3d-9706022fc01b LogTimestamp=2021-05-28T00:26:21.970400Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:26:22.291Z [qtp580718781-16] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:26:21.971Z InvocationID=684526fd-fc1a-4868-aeb2-35eb9f8a2487 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@663c9d46', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 318ms 2021-05-28T00:26:22.292Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=320 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:26:21.971Z InvocationID=684526fd-fc1a-4868-aeb2-35eb9f8a2487 ErrorCode=409 LogTimestamp=2021-05-28T00:26:22.291966Z ResponseDesc=Conflict Exiting. 2021-05-28T00:26:22.292Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:26:22.468Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:22.467929Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:22.468Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:22.467929Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:22.468Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:26:22.671Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ff593fdd-92af-425a-af3f-d51b28838b41 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:22.671442Z InvocationID=00933f60-0056-46f0-a7b6-292cf7dae724 LogTimestamp=2021-05-28T00:26:22.671487Z Entering 2021-05-28T00:26:22.671Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ff593fdd-92af-425a-af3f-d51b28838b41 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:22.671442Z InvocationID=00933f60-0056-46f0-a7b6-292cf7dae724 LogTimestamp=2021-05-28T00:26:22.671487Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:26:23.056Z [qtp580718781-12] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:26:22.672Z InvocationID=93205ba8-7816-4692-bca2-ae1a4b9066de PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@663c9d46', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 384ms 2021-05-28T00:26:23.057Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=385 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:26:22.672Z InvocationID=93205ba8-7816-4692-bca2-ae1a4b9066de ErrorCode=400 LogTimestamp=2021-05-28T00:26:23.057643Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:26:23.058Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:26:23.464Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=68634 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:25:14.830Z InvocationID=11f9984d-83a6-4223-8db2-42801908055c ErrorCode=500 LogTimestamp=2021-05-28T00:26:23.464486Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:23.465Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:26:23.758Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ddf5b1d0-0d59-4e06-a5c2-139b5f31d7f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:23.757939Z InvocationID=be49f4c0-ab3c-4fd8-b0c1-610d7866e813 LogTimestamp=2021-05-28T00:26:23.758004Z Entering 2021-05-28T00:26:23.758Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ddf5b1d0-0d59-4e06-a5c2-139b5f31d7f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:23.757939Z InvocationID=be49f4c0-ab3c-4fd8-b0c1-610d7866e813 LogTimestamp=2021-05-28T00:26:23.758004Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:26:24.181Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82a6cf60-4bb0-44f1-bf16-ade1a9c9ad5d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:24.181Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82a6cf60-4bb0-44f1-bf16-ade1a9c9ad5d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a49cad4 2021-05-28T00:26:24.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82a6cf60-4bb0-44f1-bf16-ade1a9c9ad5d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:24.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82a6cf60-4bb0-44f1-bf16-ade1a9c9ad5d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:24.182Z [qtp580718781-96] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:26:23.758Z InvocationID=70a60972-fd86-4395-8b5b-d3802c43e509 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@663c9d46', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 423ms 2021-05-28T00:26:24.183Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=425 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:26:23.758Z InvocationID=70a60972-fd86-4395-8b5b-d3802c43e509 ErrorCode=400 LogTimestamp=2021-05-28T00:26:24.183402Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:26:24.183Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:26:24.458Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:24.458842Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:24.459Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:24.458842Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:24.459Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:26:24.562Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4661ab02-73b3-41a9-9080-03877dc0093a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:24.562730Z InvocationID=7aab5a36-2509-4ed3-94b5-655dc15912c4 LogTimestamp=2021-05-28T00:26:24.562790Z Entering 2021-05-28T00:26:24.563Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4661ab02-73b3-41a9-9080-03877dc0093a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:24.562730Z InvocationID=7aab5a36-2509-4ed3-94b5-655dc15912c4 LogTimestamp=2021-05-28T00:26:24.562790Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:26:24.763Z [qtp580718781-17] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:26:24.563Z InvocationID=97ca71cc-8708-429f-be7a-6ae379cdbbaf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@663c9d46', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 199ms 2021-05-28T00:26:24.765Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=201 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:26:24.563Z InvocationID=97ca71cc-8708-429f-be7a-6ae379cdbbaf ErrorCode=409 LogTimestamp=2021-05-28T00:26:24.764793Z ResponseDesc=Conflict Exiting. 2021-05-28T00:26:24.765Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:26:24.781Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=78137 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:25:06.644Z InvocationID=82ed45a4-cdec-4c11-a152-3498ad615907 ErrorCode=500 LogTimestamp=2021-05-28T00:26:24.781677Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:24.782Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:26:24.871Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f9b3ce82-813e-436a-9f7b-5cdf9a76e2e1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:24.871152Z InvocationID=e9f31b11-3b56-4e47-aa3b-2d76ff586e42 LogTimestamp=2021-05-28T00:26:24.871203Z Entering 2021-05-28T00:26:24.871Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f9b3ce82-813e-436a-9f7b-5cdf9a76e2e1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:24.871152Z InvocationID=e9f31b11-3b56-4e47-aa3b-2d76ff586e42 LogTimestamp=2021-05-28T00:26:24.871203Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:26:25.170Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:26:24.871Z InvocationID=d9a37296-d99a-48a1-bff2-569ab6b16361 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@663c9d46', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 297ms 2021-05-28T00:26:25.170Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=299 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:26:24.871Z InvocationID=d9a37296-d99a-48a1-bff2-569ab6b16361 ErrorCode=409 LogTimestamp=2021-05-28T00:26:25.170745Z ResponseDesc=Conflict Exiting. 2021-05-28T00:26:25.171Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:26:25.272Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:25.272288Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:25.272Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:25.272288Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:25.272Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:26:25.557Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=13845b1d-a61d-4bdb-8669-e853efd90ff1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:25.557269Z InvocationID=27f6006b-5120-4681-9990-dae6cd24bd60 LogTimestamp=2021-05-28T00:26:25.557305Z Entering 2021-05-28T00:26:25.557Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=13845b1d-a61d-4bdb-8669-e853efd90ff1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:25.557269Z InvocationID=27f6006b-5120-4681-9990-dae6cd24bd60 LogTimestamp=2021-05-28T00:26:25.557305Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:26:25.682Z [qtp580718781-16] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:26:25.557Z InvocationID=3f15fa2d-f34c-41be-a733-4ede37add0fd PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@663c9d46', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 124ms 2021-05-28T00:26:25.684Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=127 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:26:25.557Z InvocationID=3f15fa2d-f34c-41be-a733-4ede37add0fd ErrorCode=409 LogTimestamp=2021-05-28T00:26:25.684503Z ResponseDesc=Conflict Exiting. 2021-05-28T00:26:25.685Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:26:26.169Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0abd10b1-6634-4157-83ff-1ea57eb5ff36 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:26.169140Z InvocationID=16b4b181-25a8-4add-ba11-812a19da850d LogTimestamp=2021-05-28T00:26:26.169197Z Entering 2021-05-28T00:26:26.169Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0abd10b1-6634-4157-83ff-1ea57eb5ff36 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:26.169140Z InvocationID=16b4b181-25a8-4add-ba11-812a19da850d LogTimestamp=2021-05-28T00:26:26.169197Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:26:26.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=148b90cb-d424-4041-a979-97b0488b60ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:26.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=148b90cb-d424-4041-a979-97b0488b60ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31be9582 2021-05-28T00:26:26.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=148b90cb-d424-4041-a979-97b0488b60ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:26.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=148b90cb-d424-4041-a979-97b0488b60ed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:26.356Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=186 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:26:26.169Z InvocationID=c4d07e27-d4ea-44b2-afd2-6ccf8e09687e LogTimestamp=2021-05-28T00:26:26.355957Z ResponseDesc=OK Exiting. 2021-05-28T00:26:26.356Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:26:29.175Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=148b90cb-d424-4041-a979-97b0488b60ed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:26:29.175Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=148b90cb-d424-4041-a979-97b0488b60ed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:26:29.176Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=148b90cb-d424-4041-a979-97b0488b60ed ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:29.176Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=148b90cb-d424-4041-a979-97b0488b60ed ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:26:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:26:31.573Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:31.572913Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:31.573Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:31.572913Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:31.573Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:26:31.861Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=069670da-8d78-4178-9c43-e79cbda2e404 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:31.860861Z InvocationID=820d7461-78bf-43b7-bfa3-5852303c6ba0 LogTimestamp=2021-05-28T00:26:31.860899Z Entering 2021-05-28T00:26:31.861Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=069670da-8d78-4178-9c43-e79cbda2e404 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:31.860861Z InvocationID=820d7461-78bf-43b7-bfa3-5852303c6ba0 LogTimestamp=2021-05-28T00:26:31.860899Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:26:31.861Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-05-28T00:26:31.861Z InvocationID=95e8f6ef-0f08-4ac9-9805-4e7732a43541 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:26:31.862Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:26:31.862Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:26:31.861Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=95e8f6ef-0f08-4ac9-9805-4e7732a43541 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@57da0153 2021-05-28T00:26:31.862Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:26:31.862Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:26:31.861Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=95e8f6ef-0f08-4ac9-9805-4e7732a43541 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@57da0153 2021-05-28T00:26:31.862Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:31.862526Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:31.862Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:31.862526Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:31.862Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="500" 2021-05-28T00:26:32.258Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:32.258237Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:32.258Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:32.258237Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:32.258Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:26:33.169Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6be1130d-3b3d-438d-81b2-8bda1909814f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:33.168729Z InvocationID=d7f6173c-82ef-4598-9036-1aefad2a1335 LogTimestamp=2021-05-28T00:26:33.168795Z Entering 2021-05-28T00:26:33.169Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6be1130d-3b3d-438d-81b2-8bda1909814f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:33.168729Z InvocationID=d7f6173c-82ef-4598-9036-1aefad2a1335 LogTimestamp=2021-05-28T00:26:33.168795Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:26:33.169Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-05-28T00:26:33.169Z InvocationID=b3600c4d-2a5d-4b80-a91f-507e8df741e6 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:26:33.170Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:26:33.170Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:26:33.169Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=b3600c4d-2a5d-4b80-a91f-507e8df741e6 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@11bf508 2021-05-28T00:26:33.170Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:26:33.170Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:26:33.169Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=b3600c4d-2a5d-4b80-a91f-507e8df741e6 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@11bf508 2021-05-28T00:26:33.170Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:33.170760Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:33.170Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:33.170760Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:33.171Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="500" 2021-05-28T00:26:33.176Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:26:33.267Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:33.267757Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:33.267Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:33.267757Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:33.268Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:26:33.290Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=77193d55-2c81-480e-b2f7-f411b6df7fbe InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:33.290327Z InvocationID=10004e2f-332a-4724-ab18-504e3f3c7da1 LogTimestamp=2021-05-28T00:26:33.290394Z Entering 2021-05-28T00:26:33.290Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=77193d55-2c81-480e-b2f7-f411b6df7fbe InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:33.290327Z InvocationID=10004e2f-332a-4724-ab18-504e3f3c7da1 LogTimestamp=2021-05-28T00:26:33.290394Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:26:33.375Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:33.290Z InvocationID=ce3ba241-c946-49d9-a542-e1a54c6f0ba7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:26:34.257Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:26:34.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=620fa49d-d346-4647-b2ee-b84b2f418f8f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:34.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=620fa49d-d346-4647-b2ee-b84b2f418f8f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69700dc7 2021-05-28T00:26:34.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=620fa49d-d346-4647-b2ee-b84b2f418f8f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:34.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=620fa49d-d346-4647-b2ee-b84b2f418f8f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:26:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:26:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:26:35.772Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=620fa49d-d346-4647-b2ee-b84b2f418f8f ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:35.772Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=620fa49d-d346-4647-b2ee-b84b2f418f8f ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:26:36.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=744d35dd-9d56-40df-8649-2e8344d5fcbf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:36.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=744d35dd-9d56-40df-8649-2e8344d5fcbf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@310e3ec 2021-05-28T00:26:36.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=744d35dd-9d56-40df-8649-2e8344d5fcbf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:36.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=744d35dd-9d56-40df-8649-2e8344d5fcbf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:26:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:26:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:26:37.871Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4581 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:33.290Z InvocationID=ce3ba241-c946-49d9-a542-e1a54c6f0ba7 LogTimestamp=2021-05-28T00:26:37.871425Z ResponseDesc=Created Exiting. 2021-05-28T00:26:37.872Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:26:38.063Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:38.063490Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:38.063Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:38.063490Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:38.064Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:26:38.171Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1fc74d83-86b7-4893-967e-fb8e8802c02e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:38.171364Z InvocationID=c3082006-3f4d-4fb8-a6fe-2f2cc106e0c1 LogTimestamp=2021-05-28T00:26:38.171403Z Entering 2021-05-28T00:26:38.171Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1fc74d83-86b7-4893-967e-fb8e8802c02e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:38.171364Z InvocationID=c3082006-3f4d-4fb8-a6fe-2f2cc106e0c1 LogTimestamp=2021-05-28T00:26:38.171403Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:26:38.778Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:38.171Z InvocationID=9aac5ee2-44c8-49f8-b5a9-ab2ecf2a4081 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:26:38.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3896a3a7-126c-4dee-b776-8f1008800b70 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:38.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3896a3a7-126c-4dee-b776-8f1008800b70 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46476489 2021-05-28T00:26:38.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3896a3a7-126c-4dee-b776-8f1008800b70 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:38.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3896a3a7-126c-4dee-b776-8f1008800b70 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:40.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbcecc8-86c7-4de8-8914-b135e7d0a0ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:40.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbcecc8-86c7-4de8-8914-b135e7d0a0ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f3057b3 2021-05-28T00:26:40.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbcecc8-86c7-4de8-8914-b135e7d0a0ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:40.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbcecc8-86c7-4de8-8914-b135e7d0a0ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:42.356Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4185 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:38.171Z InvocationID=9aac5ee2-44c8-49f8-b5a9-ab2ecf2a4081 LogTimestamp=2021-05-28T00:26:42.356360Z ResponseDesc=Created Exiting. 2021-05-28T00:26:42.357Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:26:42.472Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:42.471911Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:42.472Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:42.471911Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:42.472Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:26:42.657Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c3077915-0328-4014-a0fe-5de11809a9d0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:42.657679Z InvocationID=f61f0924-4057-4877-a355-219bb4d689f2 LogTimestamp=2021-05-28T00:26:42.657724Z Entering 2021-05-28T00:26:42.658Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c3077915-0328-4014-a0fe-5de11809a9d0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:42.657679Z InvocationID=f61f0924-4057-4877-a355-219bb4d689f2 LogTimestamp=2021-05-28T00:26:42.657724Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:26:42.768Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:42.658Z InvocationID=2fdf4672-2c8a-446e-ac12-35833107d6bb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:26:43.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49a2ec4-2a9e-490f-87f3-53624dc31a29 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:43.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49a2ec4-2a9e-490f-87f3-53624dc31a29 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14359e18 2021-05-28T00:26:43.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49a2ec4-2a9e-490f-87f3-53624dc31a29 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:43.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49a2ec4-2a9e-490f-87f3-53624dc31a29 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:48.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87268806-fd4f-47b8-b56c-c28c5ecf9679 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:48.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87268806-fd4f-47b8-b56c-c28c5ecf9679 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@103e00f1 2021-05-28T00:26:48.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87268806-fd4f-47b8-b56c-c28c5ecf9679 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:48.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87268806-fd4f-47b8-b56c-c28c5ecf9679 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:26:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:26:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:26:50.517Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:26:52.147Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:26:53.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=173dd483-1612-4a24-8565-4d10e3001706 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:53.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=173dd483-1612-4a24-8565-4d10e3001706 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6da05d53 2021-05-28T00:26:53.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=173dd483-1612-4a24-8565-4d10e3001706 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:53.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=173dd483-1612-4a24-8565-4d10e3001706 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:53.372Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=10714 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:42.658Z InvocationID=2fdf4672-2c8a-446e-ac12-35833107d6bb LogTimestamp=2021-05-28T00:26:53.372597Z ResponseDesc=Created Exiting. 2021-05-28T00:26:53.373Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:26:53.467Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:26:53.770Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:53.770394Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:53.770Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:53.770394Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:53.771Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:26:53.958Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e19c0e2a-ceec-49ad-9a66-d32ff9d7a794 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:53.958267Z InvocationID=598291fe-0494-4768-9502-1bff36b29221 LogTimestamp=2021-05-28T00:26:53.958350Z Entering 2021-05-28T00:26:53.958Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e19c0e2a-ceec-49ad-9a66-d32ff9d7a794 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:53.958267Z InvocationID=598291fe-0494-4768-9502-1bff36b29221 LogTimestamp=2021-05-28T00:26:53.958350Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:26:53.973Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:53.959Z InvocationID=db38aa09-5db9-4f25-bff2-a9b14ad2566a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:26:55.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1873a88d-b725-4e35-a507-7ac59c5f1fff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:55.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1873a88d-b725-4e35-a507-7ac59c5f1fff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4000129b 2021-05-28T00:26:55.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1873a88d-b725-4e35-a507-7ac59c5f1fff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:55.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1873a88d-b725-4e35-a507-7ac59c5f1fff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:56.862Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2903 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:53.959Z InvocationID=db38aa09-5db9-4f25-bff2-a9b14ad2566a LogTimestamp=2021-05-28T00:26:56.862659Z ResponseDesc=Created Exiting. 2021-05-28T00:26:56.863Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:26:56.877Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:56.877099Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:26:56.877Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:26:56.877099Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:26:56.877Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:26:56.958Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=de430b3f-a593-45e9-8e2e-4dac2aa68d2d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:56.958339Z InvocationID=2f1b499d-a7c4-44eb-a72f-7dd64374c50a LogTimestamp=2021-05-28T00:26:56.958397Z Entering 2021-05-28T00:26:56.958Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=de430b3f-a593-45e9-8e2e-4dac2aa68d2d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:26:56.958339Z InvocationID=2f1b499d-a7c4-44eb-a72f-7dd64374c50a LogTimestamp=2021-05-28T00:26:56.958397Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:26:56.975Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:56.959Z InvocationID=25a04959-2db4-44f9-aada-4ad53c3845f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:26:57.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:26:57.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78fd81dd 2021-05-28T00:26:57.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:26:57.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:26:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:26:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:27:00.282Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:27:00.283Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:27:00.283Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:27:00.283Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0aac911c-ccb1-4e93-a078-83c36e433778 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:27:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:27:03.460Z [qtp580718781-15] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:56.959Z InvocationID=25a04959-2db4-44f9-aada-4ad53c3845f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Temporary storage failure while checking locks org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.getSliceWithRetries(ConsistentKeyLocker.java:464) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:426) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:175) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:154) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:72) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:279) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1265) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1249) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:52) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:115) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:96) at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:75) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:76) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency QUORUM (2 responses were required but only 0 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency QUORUM (2 responses were required but only 0 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency QUORUM (2 responses were required but only 0 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2021-05-28T00:27:04.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb223ae1-73b7-490a-a37f-90f4e249eaec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:04.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb223ae1-73b7-490a-a37f-90f4e249eaec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18a43fbb 2021-05-28T00:27:04.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb223ae1-73b7-490a-a37f-90f4e249eaec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:04.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb223ae1-73b7-490a-a37f-90f4e249eaec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:04.664Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:27:04.982Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=bb223ae1-73b7-490a-a37f-90f4e249eaec ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:27:04.982Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=bb223ae1-73b7-490a-a37f-90f4e249eaec ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:27:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:27:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:27:05.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa13ab3-49c7-4ea7-a278-2a71a76ff226 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:05.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa13ab3-49c7-4ea7-a278-2a71a76ff226 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3755e398 2021-05-28T00:27:05.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa13ab3-49c7-4ea7-a278-2a71a76ff226 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:05.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa13ab3-49c7-4ea7-a278-2a71a76ff226 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:06.375Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=9415 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:26:56.959Z InvocationID=25a04959-2db4-44f9-aada-4ad53c3845f0 LogTimestamp=2021-05-28T00:27:06.374085Z ResponseDesc=Created Exiting. 2021-05-28T00:27:06.375Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:06.709Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:06.709468Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:06.709Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:06.709468Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:06.709Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:27:06.763Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7fa13ab3-49c7-4ea7-a278-2a71a76ff226 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:27:06.763Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7fa13ab3-49c7-4ea7-a278-2a71a76ff226 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:27:06.960Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2432bb43-d235-4f6d-93e8-fb9609c57cab InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:06.959419Z InvocationID=9bdece64-df8c-41d6-9246-ccab0162cfd3 LogTimestamp=2021-05-28T00:27:06.959505Z Entering 2021-05-28T00:27:06.960Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2432bb43-d235-4f6d-93e8-fb9609c57cab InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:06.959419Z InvocationID=9bdece64-df8c-41d6-9246-ccab0162cfd3 LogTimestamp=2021-05-28T00:27:06.959505Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:06.978Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:06.960Z InvocationID=a6b71695-743d-46c7-9f10-601dd107f98f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:27:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fc4ce32-6b0b-44eb-a148-cbfa48f6c4d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fc4ce32-6b0b-44eb-a148-cbfa48f6c4d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23327fc6 2021-05-28T00:27:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fc4ce32-6b0b-44eb-a148-cbfa48f6c4d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:07.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fc4ce32-6b0b-44eb-a148-cbfa48f6c4d0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:09.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2064443-b19a-4a12-a77d-849103ca33ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:09.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2064443-b19a-4a12-a77d-849103ca33ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a52d5fc 2021-05-28T00:27:09.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2064443-b19a-4a12-a77d-849103ca33ce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:09.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2064443-b19a-4a12-a77d-849103ca33ce PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:09.673Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2713 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:06.960Z InvocationID=a6b71695-743d-46c7-9f10-601dd107f98f LogTimestamp=2021-05-28T00:27:09.673435Z ResponseDesc=Created Exiting. 2021-05-28T00:27:09.674Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:09.775Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:09.775892Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:09.776Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:09.775892Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:09.776Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:27:09.880Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5c714149-b8ba-40e9-be63-467ffda85300 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:09.880247Z InvocationID=e9b24d54-e006-4754-a526-f9b2ab1fa9f0 LogTimestamp=2021-05-28T00:27:09.880290Z Entering 2021-05-28T00:27:09.880Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5c714149-b8ba-40e9-be63-467ffda85300 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:09.880247Z InvocationID=e9b24d54-e006-4754-a526-f9b2ab1fa9f0 LogTimestamp=2021-05-28T00:27:09.880290Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:09.980Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:09.880Z InvocationID=d1d22c69-a58c-481c-aba3-c375859a9550 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:27:10.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cdc0045-6b88-4ba3-8aea-8669934f9eac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:10.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cdc0045-6b88-4ba3-8aea-8669934f9eac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6030a5f4 2021-05-28T00:27:10.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cdc0045-6b88-4ba3-8aea-8669934f9eac PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:10.485Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cdc0045-6b88-4ba3-8aea-8669934f9eac PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:12.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bba2a1-f458-416b-aabc-cf26e195f873 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:12.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bba2a1-f458-416b-aabc-cf26e195f873 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78e1d349 2021-05-28T00:27:12.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bba2a1-f458-416b-aabc-cf26e195f873 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:12.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8bba2a1-f458-416b-aabc-cf26e195f873 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:14.075Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4195 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:09.880Z InvocationID=d1d22c69-a58c-481c-aba3-c375859a9550 LogTimestamp=2021-05-28T00:27:14.075259Z ResponseDesc=Created Exiting. 2021-05-28T00:27:14.075Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:14.260Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:14.260131Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:14.260Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:14.260131Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:14.260Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:27:14.374Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=50750068-3a15-45f2-ab45-5004b615ed30 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:14.374191Z InvocationID=7a076963-3c81-4dd9-8aa8-354f8ac2a4f6 LogTimestamp=2021-05-28T00:27:14.374240Z Entering 2021-05-28T00:27:14.374Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=50750068-3a15-45f2-ab45-5004b615ed30 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:14.374191Z InvocationID=7a076963-3c81-4dd9-8aa8-354f8ac2a4f6 LogTimestamp=2021-05-28T00:27:14.374240Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:14.568Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:14.374Z InvocationID=a61b8be3-36c0-46a0-b3fd-10130184fcdf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:27:14.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c923e0a7-20bc-44b5-b43d-ed70beb4bc52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:14.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c923e0a7-20bc-44b5-b43d-ed70beb4bc52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@481d5fdc 2021-05-28T00:27:14.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c923e0a7-20bc-44b5-b43d-ed70beb4bc52 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:14.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c923e0a7-20bc-44b5-b43d-ed70beb4bc52 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:16.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29961f0d-1303-4b52-8c14-10bfcdea039d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:16.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29961f0d-1303-4b52-8c14-10bfcdea039d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44ec9e16 2021-05-28T00:27:16.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29961f0d-1303-4b52-8c14-10bfcdea039d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:16.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29961f0d-1303-4b52-8c14-10bfcdea039d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:17.291Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2917 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:14.374Z InvocationID=a61b8be3-36c0-46a0-b3fd-10130184fcdf LogTimestamp=2021-05-28T00:27:17.291582Z ResponseDesc=Created Exiting. 2021-05-28T00:27:17.292Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:17.471Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:17.471530Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:17.471Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:17.471530Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:17.472Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:27:17.485Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d0af34a0-70d0-4a72-a6b4-4b81a9954aad InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:17.485679Z InvocationID=e9402189-fae8-4713-81ef-94e95d83e0f2 LogTimestamp=2021-05-28T00:27:17.485732Z Entering 2021-05-28T00:27:17.486Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d0af34a0-70d0-4a72-a6b4-4b81a9954aad InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:17.485679Z InvocationID=e9402189-fae8-4713-81ef-94e95d83e0f2 LogTimestamp=2021-05-28T00:27:17.485732Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:17.677Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:17.486Z InvocationID=db408bde-d197-4241-b55d-42f0c91558d5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:27:20.077Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2591 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:17.486Z InvocationID=db408bde-d197-4241-b55d-42f0c91558d5 LogTimestamp=2021-05-28T00:27:20.077026Z ResponseDesc=Created Exiting. 2021-05-28T00:27:20.077Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:20.257Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=13585 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:27:06.672Z InvocationID=1ce06555-cb1e-47a7-9b15-3501e912b666 ErrorCode=500 LogTimestamp=2021-05-28T00:27:20.257189Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:20.258Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:27:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:27:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:27:20.680Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b19b98a8-d14b-4050-8fac-f3e157d4cc73 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:20.679764Z InvocationID=4ecdcf81-a777-4a2e-acb1-9c637c531cda LogTimestamp=2021-05-28T00:27:20.679821Z Entering 2021-05-28T00:27:20.680Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b19b98a8-d14b-4050-8fac-f3e157d4cc73 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:20.679764Z InvocationID=4ecdcf81-a777-4a2e-acb1-9c637c531cda LogTimestamp=2021-05-28T00:27:20.679821Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:20.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f554687-c184-48d6-a4c4-7ea7ef049ac2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:20.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f554687-c184-48d6-a4c4-7ea7ef049ac2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21968c7a 2021-05-28T00:27:20.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f554687-c184-48d6-a4c4-7ea7ef049ac2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:20.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f554687-c184-48d6-a4c4-7ea7ef049ac2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:20.873Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:20.680Z InvocationID=cadb4cd7-2833-448c-a1d7-cd241ea96d0e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:27:23.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1788aeac-49bd-4a0e-b74b-e0a9e3188a3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:23.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1788aeac-49bd-4a0e-b74b-e0a9e3188a3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@579b689a 2021-05-28T00:27:23.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1788aeac-49bd-4a0e-b74b-e0a9e3188a3f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:23.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1788aeac-49bd-4a0e-b74b-e0a9e3188a3f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:25.781Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=5101 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:20.680Z InvocationID=cadb4cd7-2833-448c-a1d7-cd241ea96d0e LogTimestamp=2021-05-28T00:27:25.781276Z ResponseDesc=Created Exiting. 2021-05-28T00:27:25.782Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:25.970Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:25.970875Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:25.971Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:25.970875Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:25.972Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:27:26.071Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f756f302-daf6-4163-946e-b115380ffa59 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:26.071460Z InvocationID=1b7b9cb7-768d-43e5-b59b-a44f6dcc839d LogTimestamp=2021-05-28T00:27:26.071507Z Entering 2021-05-28T00:27:26.072Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f756f302-daf6-4163-946e-b115380ffa59 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:26.071460Z InvocationID=1b7b9cb7-768d-43e5-b59b-a44f6dcc839d LogTimestamp=2021-05-28T00:27:26.071507Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:26.159Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:26.072Z InvocationID=779c0fc8-bd85-43c8-8392-727c2278d9d1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:27:26.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24d8db-02ff-441f-beda-954f997e68cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:26.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24d8db-02ff-441f-beda-954f997e68cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75da3e0a 2021-05-28T00:27:26.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24d8db-02ff-441f-beda-954f997e68cd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:26.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24d8db-02ff-441f-beda-954f997e68cd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fe27b2b-9c3f-4f6d-905b-72338f9849e8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fe27b2b-9c3f-4f6d-905b-72338f9849e8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55e61289 2021-05-28T00:27:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fe27b2b-9c3f-4f6d-905b-72338f9849e8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fe27b2b-9c3f-4f6d-905b-72338f9849e8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:28.771Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2698 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:26.072Z InvocationID=779c0fc8-bd85-43c8-8392-727c2278d9d1 LogTimestamp=2021-05-28T00:27:28.770823Z ResponseDesc=Created Exiting. 2021-05-28T00:27:28.771Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:28.783Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:28.783299Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:28.783Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:28.783299Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:28.783Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:27:28.983Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=56d2b3ef-1d16-49cb-b437-d0a52a94c2c9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:28.982691Z InvocationID=dc5a03f2-1fd8-4262-800d-3ceef3d14f13 LogTimestamp=2021-05-28T00:27:28.982753Z Entering 2021-05-28T00:27:28.983Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=56d2b3ef-1d16-49cb-b437-d0a52a94c2c9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:28.982691Z InvocationID=dc5a03f2-1fd8-4262-800d-3ceef3d14f13 LogTimestamp=2021-05-28T00:27:28.982753Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:29.171Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:28.983Z InvocationID=092850e0-3495-415a-bf99-919d7a2a6ab0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:27:30.856Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1873 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:28.983Z InvocationID=092850e0-3495-415a-bf99-919d7a2a6ab0 LogTimestamp=2021-05-28T00:27:30.856150Z ResponseDesc=Created Exiting. 2021-05-28T00:27:30.856Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:30.957Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:30.957365Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:30.957Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:30.957365Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:30.958Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:27:30.971Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3fb73efb-4d11-4508-8487-23bf3f1cc563 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:30.971479Z InvocationID=839aa9ff-4920-412f-b63a-081ec8605408 LogTimestamp=2021-05-28T00:27:30.971510Z Entering 2021-05-28T00:27:30.971Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3fb73efb-4d11-4508-8487-23bf3f1cc563 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:30.971479Z InvocationID=839aa9ff-4920-412f-b63a-081ec8605408 LogTimestamp=2021-05-28T00:27:30.971510Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:27:31.071Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:30.971Z InvocationID=4149af44-62ca-4425-9432-8b51819c4eaa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:27:31.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe72cd14-739e-4074-bb9b-6b8d855e4404 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:31.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe72cd14-739e-4074-bb9b-6b8d855e4404 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4086a113 2021-05-28T00:27:31.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe72cd14-739e-4074-bb9b-6b8d855e4404 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:31.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe72cd14-739e-4074-bb9b-6b8d855e4404 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:33.466Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2495 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:27:30.971Z InvocationID=4149af44-62ca-4425-9432-8b51819c4eaa LogTimestamp=2021-05-28T00:27:33.466253Z ResponseDesc=Created Exiting. 2021-05-28T00:27:33.467Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:27:33.960Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=67d0d4fa-29e6-4fc3-83ec-f395f58aa227 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:33.960369Z InvocationID=e4380e6e-e214-45d5-b5d6-7b6e67bcd938 LogTimestamp=2021-05-28T00:27:33.960443Z Entering 2021-05-28T00:27:33.960Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=67d0d4fa-29e6-4fc3-83ec-f395f58aa227 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:33.960369Z InvocationID=e4380e6e-e214-45d5-b5d6-7b6e67bcd938 LogTimestamp=2021-05-28T00:27:33.960443Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:27:34.068Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=107 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:27:33.961Z InvocationID=df5930ec-f59f-49d0-a144-5ad8c1789794 LogTimestamp=2021-05-28T00:27:34.068188Z ResponseDesc=OK Exiting. 2021-05-28T00:27:34.068Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:27:34.880Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ef8bc122-7777-4a62-9a92-ca9fff735f46 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:34.880403Z InvocationID=218cd6e6-4e1f-4a48-be99-3b0417b90d7f LogTimestamp=2021-05-28T00:27:34.880441Z Entering 2021-05-28T00:27:34.880Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ef8bc122-7777-4a62-9a92-ca9fff735f46 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:34.880403Z InvocationID=218cd6e6-4e1f-4a48-be99-3b0417b90d7f LogTimestamp=2021-05-28T00:27:34.880441Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:27:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:27:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:27:35.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6f8e39e-085d-4b3c-8186-809d5d10aa62 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:35.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6f8e39e-085d-4b3c-8186-809d5d10aa62 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4cd5ce21 2021-05-28T00:27:35.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6f8e39e-085d-4b3c-8186-809d5d10aa62 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:35.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6f8e39e-085d-4b3c-8186-809d5d10aa62 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:35.575Z [qtp580718781-15] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:27:34.881Z InvocationID=dcaafefb-9e0d-47a6-9f57-4f15761dbc5d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 692214498 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n692214498 100% \n 2021-05-28T00:27:35.584Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=703 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:27:34.881Z InvocationID=dcaafefb-9e0d-47a6-9f57-4f15761dbc5d LogTimestamp=2021-05-28T00:27:35.584067Z ResponseDesc=OK Exiting. 2021-05-28T00:27:35.584Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:27:35.771Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=278a83b6-5c5a-406d-a246-2c4c64ac6e9d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:35.771497Z InvocationID=184eee93-3369-4a49-a9cc-d1c5f889740e LogTimestamp=2021-05-28T00:27:35.771559Z Entering 2021-05-28T00:27:35.771Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=278a83b6-5c5a-406d-a246-2c4c64ac6e9d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:35.771497Z InvocationID=184eee93-3369-4a49-a9cc-d1c5f889740e LogTimestamp=2021-05-28T00:27:35.771559Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:27:35.902Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:27:35.772Z InvocationID=e1637ce0-6223-48e6-a147-6d41eb7b510b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:27:35.999Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:27:35.772Z InvocationID=e1637ce0-6223-48e6-a147-6d41eb7b510b ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:27:35.999Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:27:35.772Z InvocationID=e1637ce0-6223-48e6-a147-6d41eb7b510b ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:27:36.000Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=228 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:27:35.772Z InvocationID=e1637ce0-6223-48e6-a147-6d41eb7b510b ErrorCode=500 LogTimestamp=2021-05-28T00:27:36.000225Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:36.000Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:27:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:27:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:27:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:27:37.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc27a16f-7c0a-4b08-92a7-f9c411ebbf7e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:37.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc27a16f-7c0a-4b08-92a7-f9c411ebbf7e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68db93df 2021-05-28T00:27:37.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc27a16f-7c0a-4b08-92a7-f9c411ebbf7e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:37.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc27a16f-7c0a-4b08-92a7-f9c411ebbf7e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:40.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5842c434-fa57-46d1-a912-e7f4cbd1c945 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:40.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5842c434-fa57-46d1-a912-e7f4cbd1c945 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d4ca617 2021-05-28T00:27:40.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5842c434-fa57-46d1-a912-e7f4cbd1c945 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:40.890Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5842c434-fa57-46d1-a912-e7f4cbd1c945 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:43.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf9c56f3-19b5-4421-a400-192133cd57ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:43.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf9c56f3-19b5-4421-a400-192133cd57ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78b71094 2021-05-28T00:27:43.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf9c56f3-19b5-4421-a400-192133cd57ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:43.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf9c56f3-19b5-4421-a400-192133cd57ed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:46.173Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3b6f58ae-4344-4eb2-8ef6-773013b959ef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:46.173166Z InvocationID=01f6554f-bae1-4850-be9d-efb58cc416ac LogTimestamp=2021-05-28T00:27:46.173290Z Entering 2021-05-28T00:27:46.173Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3b6f58ae-4344-4eb2-8ef6-773013b959ef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:46.173166Z InvocationID=01f6554f-bae1-4850-be9d-efb58cc416ac LogTimestamp=2021-05-28T00:27:46.173290Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:27:46.270Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:27:46.174Z InvocationID=1338777d-0e40-4b0b-9f24-f02404f6d7e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:27:46.367Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:27:46.174Z InvocationID=1338777d-0e40-4b0b-9f24-f02404f6d7e6 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:27:46.368Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:27:46.174Z InvocationID=1338777d-0e40-4b0b-9f24-f02404f6d7e6 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:27:46.368Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=194 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:27:46.174Z InvocationID=1338777d-0e40-4b0b-9f24-f02404f6d7e6 ErrorCode=500 LogTimestamp=2021-05-28T00:27:46.368769Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:46.369Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:27:46.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4fe10d-4d73-40b4-92a3-ce851ba6ae6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:46.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4fe10d-4d73-40b4-92a3-ce851ba6ae6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a717d65 2021-05-28T00:27:46.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4fe10d-4d73-40b4-92a3-ce851ba6ae6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:46.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4fe10d-4d73-40b4-92a3-ce851ba6ae6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:50.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:27:50.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e00509f 2021-05-28T00:27:50.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:27:50.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:27:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:27:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:27:52.152Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:27:53.375Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:27:53.376Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:27:53.376Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:27:53.376Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6217d606-21ea-4584-89c2-3ae35e8cca9d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:27:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:27:53.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:27:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:27:58.382Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:27:58.459Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:27:58.968Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4fc1658a-af87-46db-8745-5e5338201cbb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:58.968451Z InvocationID=f714428e-27e7-4307-8ee5-b41b3e121632 LogTimestamp=2021-05-28T00:27:58.968534Z Entering 2021-05-28T00:27:58.969Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4fc1658a-af87-46db-8745-5e5338201cbb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:27:58.968451Z InvocationID=f714428e-27e7-4307-8ee5-b41b3e121632 LogTimestamp=2021-05-28T00:27:58.968534Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:27:58.969Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:27:58.969Z InvocationID=41801a2d-76b0-416a-be38-fe28f542db37 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:27:58.970Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:27:58.970Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:27:58.969Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=41801a2d-76b0-416a-be38-fe28f542db37 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@28a92e73 2021-05-28T00:27:58.970Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:27:58.970Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:27:58.969Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=41801a2d-76b0-416a-be38-fe28f542db37 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@28a92e73 2021-05-28T00:27:58.970Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:58.970686Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:27:58.970Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:27:58.970686Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:27:58.971Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:27:59.674Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:28:00.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30db7229-2edc-4218-8ac7-fd1b95a01f5c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:00.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30db7229-2edc-4218-8ac7-fd1b95a01f5c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46143756 2021-05-28T00:28:00.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30db7229-2edc-4218-8ac7-fd1b95a01f5c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:00.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30db7229-2edc-4218-8ac7-fd1b95a01f5c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:01.079Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=30db7229-2edc-4218-8ac7-fd1b95a01f5c ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:01.080Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=30db7229-2edc-4218-8ac7-fd1b95a01f5c ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:04.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1f5346e-c621-4f73-9af3-75133a3fa840 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:04.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1f5346e-c621-4f73-9af3-75133a3fa840 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17ec635c 2021-05-28T00:28:04.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1f5346e-c621-4f73-9af3-75133a3fa840 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:04.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1f5346e-c621-4f73-9af3-75133a3fa840 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:28:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:28:06.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=129db34e-0cbf-42e5-89e9-7ff06299b90d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:06.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=129db34e-0cbf-42e5-89e9-7ff06299b90d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b444d96 2021-05-28T00:28:06.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=129db34e-0cbf-42e5-89e9-7ff06299b90d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:06.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=129db34e-0cbf-42e5-89e9-7ff06299b90d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:08.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c95fd837-5045-4ca6-987a-932db1cc1c1a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:08.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c95fd837-5045-4ca6-987a-932db1cc1c1a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c0d632f 2021-05-28T00:28:08.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c95fd837-5045-4ca6-987a-932db1cc1c1a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:08.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c95fd837-5045-4ca6-987a-932db1cc1c1a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:09.086Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=890b2242-a12f-4474-84b2-b1d11f3704e8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:09.086408Z InvocationID=61f8a739-6de6-4b12-97b4-b88684b5ddb9 LogTimestamp=2021-05-28T00:28:09.086533Z Entering 2021-05-28T00:28:09.087Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=890b2242-a12f-4474-84b2-b1d11f3704e8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:09.086408Z InvocationID=61f8a739-6de6-4b12-97b4-b88684b5ddb9 LogTimestamp=2021-05-28T00:28:09.086533Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:28:09.202Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:28:09.087Z InvocationID=a086c84e-bdab-4f49-a53b-65f940c4a386 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:28:09.274Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:09.087Z InvocationID=a086c84e-bdab-4f49-a53b-65f940c4a386 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:28:09.274Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:09.087Z InvocationID=a086c84e-bdab-4f49-a53b-65f940c4a386 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:28:09.275Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=188 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:28:09.087Z InvocationID=a086c84e-bdab-4f49-a53b-65f940c4a386 ErrorCode=500 LogTimestamp=2021-05-28T00:28:09.275272Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:28:09.275Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:28:09.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d65df67-8bb5-4892-9f68-4a9233c3ef91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:09.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d65df67-8bb5-4892-9f68-4a9233c3ef91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67b44622 2021-05-28T00:28:09.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d65df67-8bb5-4892-9f68-4a9233c3ef91 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:09.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d65df67-8bb5-4892-9f68-4a9233c3ef91 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:10.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83dc25b1-f87e-4fd5-a955-ca427d3c6d1a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:10.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83dc25b1-f87e-4fd5-a955-ca427d3c6d1a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13363f0b 2021-05-28T00:28:10.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83dc25b1-f87e-4fd5-a955-ca427d3c6d1a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:10.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83dc25b1-f87e-4fd5-a955-ca427d3c6d1a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:12.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e05f288f-1031-4f40-91d3-7dac87bfe75c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:12.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e05f288f-1031-4f40-91d3-7dac87bfe75c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a07e806 2021-05-28T00:28:12.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e05f288f-1031-4f40-91d3-7dac87bfe75c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:12.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e05f288f-1031-4f40-91d3-7dac87bfe75c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:14.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30024c49-fdbd-443d-b0e4-b0a9735fd161 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:14.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30024c49-fdbd-443d-b0e4-b0a9735fd161 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3da4e672 2021-05-28T00:28:14.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30024c49-fdbd-443d-b0e4-b0a9735fd161 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:14.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30024c49-fdbd-443d-b0e4-b0a9735fd161 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:16.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96cc5b9e-1c96-49e4-9c49-d2039bf452de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:16.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96cc5b9e-1c96-49e4-9c49-d2039bf452de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78e422b2 2021-05-28T00:28:16.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96cc5b9e-1c96-49e4-9c49-d2039bf452de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:16.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96cc5b9e-1c96-49e4-9c49-d2039bf452de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:19.395Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0587ae8b-b72f-4c66-85f3-561e5603a1f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:19.395322Z InvocationID=7d727030-8deb-4097-bd9d-954bd5551746 LogTimestamp=2021-05-28T00:28:19.395413Z Entering 2021-05-28T00:28:19.395Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0587ae8b-b72f-4c66-85f3-561e5603a1f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:19.395322Z InvocationID=7d727030-8deb-4097-bd9d-954bd5551746 LogTimestamp=2021-05-28T00:28:19.395413Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:28:19.516Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:28:19.396Z InvocationID=f63add90-a142-4951-8ec4-0382f81b6edf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:28:19.685Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:19.396Z InvocationID=f63add90-a142-4951-8ec4-0382f81b6edf ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:28:19.686Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:19.396Z InvocationID=f63add90-a142-4951-8ec4-0382f81b6edf ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:28:19.687Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=291 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:28:19.396Z InvocationID=f63add90-a142-4951-8ec4-0382f81b6edf ErrorCode=500 LogTimestamp=2021-05-28T00:28:19.687186Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:28:19.687Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:28:20.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8138de52-73c6-4739-991d-aae06f01dad0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:20.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8138de52-73c6-4739-991d-aae06f01dad0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a8884c2 2021-05-28T00:28:20.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8138de52-73c6-4739-991d-aae06f01dad0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:20.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8138de52-73c6-4739-991d-aae06f01dad0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:28:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:28:23.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3ce763-9c01-47f9-991d-d98149a6c638 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:23.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3ce763-9c01-47f9-991d-d98149a6c638 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ffa004d 2021-05-28T00:28:23.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3ce763-9c01-47f9-991d-d98149a6c638 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:23.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3ce763-9c01-47f9-991d-d98149a6c638 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3e28836-33e3-4d59-b153-009f1eaf6bcf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3e28836-33e3-4d59-b153-009f1eaf6bcf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75f7fcb6 2021-05-28T00:28:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3e28836-33e3-4d59-b153-009f1eaf6bcf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3e28836-33e3-4d59-b153-009f1eaf6bcf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5622e82-c8a3-4aa6-bb6d-ca3f3fb8c190 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5622e82-c8a3-4aa6-bb6d-ca3f3fb8c190 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c5fb108 2021-05-28T00:28:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5622e82-c8a3-4aa6-bb6d-ca3f3fb8c190 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5622e82-c8a3-4aa6-bb6d-ca3f3fb8c190 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:29.779Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5dc5e9e8-55ff-4395-b84b-1f3baf091e43 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:29.778988Z InvocationID=e7e8277e-b7a0-4426-bd2f-9e5330bba3bc LogTimestamp=2021-05-28T00:28:29.779057Z Entering 2021-05-28T00:28:29.779Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5dc5e9e8-55ff-4395-b84b-1f3baf091e43 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:29.778988Z InvocationID=e7e8277e-b7a0-4426-bd2f-9e5330bba3bc LogTimestamp=2021-05-28T00:28:29.779057Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:28:29.869Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:28:29.779Z InvocationID=4f79a926-d515-4706-9bce-a3266b0c7559 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:28:29.878Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:29.779Z InvocationID=4f79a926-d515-4706-9bce-a3266b0c7559 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:28:29.879Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:29.779Z InvocationID=4f79a926-d515-4706-9bce-a3266b0c7559 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:28:29.879Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=100 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:28:29.779Z InvocationID=4f79a926-d515-4706-9bce-a3266b0c7559 ErrorCode=500 LogTimestamp=2021-05-28T00:28:29.879573Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:28:29.880Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:28:33.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadefdbe-90a3-4136-b158-4050fa09a6ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:33.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadefdbe-90a3-4136-b158-4050fa09a6ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@326f8ed5 2021-05-28T00:28:33.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadefdbe-90a3-4136-b158-4050fa09a6ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:33.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadefdbe-90a3-4136-b158-4050fa09a6ec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:34.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1efeb25d-09cc-4f0d-a58f-7675aada204f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:34.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1efeb25d-09cc-4f0d-a58f-7675aada204f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@da808bd 2021-05-28T00:28:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1efeb25d-09cc-4f0d-a58f-7675aada204f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1efeb25d-09cc-4f0d-a58f-7675aada204f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:28:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:28:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:28:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:28:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:28:38.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11d15961-e562-4c07-8621-670e8fdfc592 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:38.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11d15961-e562-4c07-8621-670e8fdfc592 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@477047f2 2021-05-28T00:28:38.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11d15961-e562-4c07-8621-670e8fdfc592 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:38.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11d15961-e562-4c07-8621-670e8fdfc592 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:39.976Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c03057e8-3ec4-436a-974d-96fb46c4d12d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:39.976415Z InvocationID=e6380f16-1b65-476e-8c29-547d77e503a4 LogTimestamp=2021-05-28T00:28:39.976526Z Entering 2021-05-28T00:28:39.977Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c03057e8-3ec4-436a-974d-96fb46c4d12d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:39.976415Z InvocationID=e6380f16-1b65-476e-8c29-547d77e503a4 LogTimestamp=2021-05-28T00:28:39.976526Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:28:40.075Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:28:39.977Z InvocationID=97b3f754-2236-4f70-aae2-6a280be296b1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:28:40.282Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:39.977Z InvocationID=97b3f754-2236-4f70-aae2-6a280be296b1 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:28:40.283Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:39.977Z InvocationID=97b3f754-2236-4f70-aae2-6a280be296b1 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:28:40.284Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=307 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:28:39.977Z InvocationID=97b3f754-2236-4f70-aae2-6a280be296b1 ErrorCode=500 LogTimestamp=2021-05-28T00:28:40.284151Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:28:40.284Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:28:40.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb211145-361a-47b7-b72d-29a521d75d97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:40.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb211145-361a-47b7-b72d-29a521d75d97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@167669e5 2021-05-28T00:28:40.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb211145-361a-47b7-b72d-29a521d75d97 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:40.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb211145-361a-47b7-b72d-29a521d75d97 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:43.991Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=fb211145-361a-47b7-b72d-29a521d75d97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:28:43.992Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=fb211145-361a-47b7-b72d-29a521d75d97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:28:43.992Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fb211145-361a-47b7-b72d-29a521d75d97 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:43.992Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fb211145-361a-47b7-b72d-29a521d75d97 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:46.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ad4072d-1107-42c4-8711-1be33cd83178 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:46.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ad4072d-1107-42c4-8711-1be33cd83178 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58a144d8 2021-05-28T00:28:46.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ad4072d-1107-42c4-8711-1be33cd83178 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:46.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ad4072d-1107-42c4-8711-1be33cd83178 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:46.886Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2ad4072d-1107-42c4-8711-1be33cd83178 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:46.886Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2ad4072d-1107-42c4-8711-1be33cd83178 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:48.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b303e472-d6a8-43e3-99d9-a9b7d49bb5c9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:48.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b303e472-d6a8-43e3-99d9-a9b7d49bb5c9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@500bc804 2021-05-28T00:28:48.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b303e472-d6a8-43e3-99d9-a9b7d49bb5c9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:48.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b303e472-d6a8-43e3-99d9-a9b7d49bb5c9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:28:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7517baa1 2021-05-28T00:28:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:28:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:28:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:28:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:28:52.325Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:28:52.878Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:28:52.879Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:28:52.879Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:52.879Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=69f9e377-8ff2-47ab-8720-2379642d76c7 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:28:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:28:56.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:28:57.294Z [qtp580718781-17] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.
(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator. (ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:674) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:347) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:136) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:80) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:72) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:74) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:85) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:80) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:75) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 0 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 0 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 0 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:28:57.386Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3c8a2787-937e-4120-bea6-3ed05ec3d877 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:57.386678Z InvocationID=ce2d8d74-31bb-403f-8b80-6da7e0c63005 LogTimestamp=2021-05-28T00:28:57.386727Z Entering 2021-05-28T00:28:57.387Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3c8a2787-937e-4120-bea6-3ed05ec3d877 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:28:57.386678Z InvocationID=ce2d8d74-31bb-403f-8b80-6da7e0c63005 LogTimestamp=2021-05-28T00:28:57.386727Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:28:57.387Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:28:57.387Z InvocationID=a258b8a1-35ad-4e7b-8cdf-431d21bd9ef3 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:28:57.388Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:28:57.388Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:28:57.387Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=a258b8a1-35ad-4e7b-8cdf-431d21bd9ef3 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@692b201b 2021-05-28T00:28:57.388Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:28:57.388Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:28:57.387Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=a258b8a1-35ad-4e7b-8cdf-431d21bd9ef3 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@692b201b 2021-05-28T00:28:57.389Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:28:57.389119Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:28:57.389Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:28:57.389119Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:28:57.389Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:28:57.966Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:28:58.485Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:29:02.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f8d12da-4cdd-4157-a047-8788be3aebbb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:02.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f8d12da-4cdd-4157-a047-8788be3aebbb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27c2b75f 2021-05-28T00:29:02.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f8d12da-4cdd-4157-a047-8788be3aebbb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:02.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f8d12da-4cdd-4157-a047-8788be3aebbb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:02.577Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4f8d12da-4cdd-4157-a047-8788be3aebbb ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:02.577Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4f8d12da-4cdd-4157-a047-8788be3aebbb ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:05.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5967b025-7282-4a29-92ec-93b99ff511a7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:05.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5967b025-7282-4a29-92ec-93b99ff511a7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@368d5809 2021-05-28T00:29:05.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5967b025-7282-4a29-92ec-93b99ff511a7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:05.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5967b025-7282-4a29-92ec-93b99ff511a7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:29:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:29:06.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682d5898-dadc-4bf2-965b-f30ae34e1761 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:06.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682d5898-dadc-4bf2-965b-f30ae34e1761 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e873ecb 2021-05-28T00:29:06.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682d5898-dadc-4bf2-965b-f30ae34e1761 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:06.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=682d5898-dadc-4bf2-965b-f30ae34e1761 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:07.475Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=998f35f6-b2ff-49df-be7b-bdaff243561d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:29:07.475217Z InvocationID=089e5cd5-1eb9-4c74-8848-5e38fe1486ba LogTimestamp=2021-05-28T00:29:07.475276Z Entering 2021-05-28T00:29:07.475Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=998f35f6-b2ff-49df-be7b-bdaff243561d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:29:07.475217Z InvocationID=089e5cd5-1eb9-4c74-8848-5e38fe1486ba LogTimestamp=2021-05-28T00:29:07.475276Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:29:07.746Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:29:07.476Z InvocationID=8ec1a6b6-31ee-4fb6-a1e7-a61fe8650fc4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:29:07.772Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:29:07.476Z InvocationID=8ec1a6b6-31ee-4fb6-a1e7-a61fe8650fc4 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:29:07.772Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:29:07.476Z InvocationID=8ec1a6b6-31ee-4fb6-a1e7-a61fe8650fc4 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:29:07.773Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=297 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:29:07.476Z InvocationID=8ec1a6b6-31ee-4fb6-a1e7-a61fe8650fc4 ErrorCode=500 LogTimestamp=2021-05-28T00:29:07.773212Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:29:07.773Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:29:08.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=925f63cc-83f5-44c2-ac71-35be8f7086db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:08.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=925f63cc-83f5-44c2-ac71-35be8f7086db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ccd060b 2021-05-28T00:29:08.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=925f63cc-83f5-44c2-ac71-35be8f7086db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:08.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=925f63cc-83f5-44c2-ac71-35be8f7086db PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:10.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:10.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@339dac62 2021-05-28T00:29:10.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:10.466Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:13.471Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:29:13.472Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:29:13.472Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:13.472Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e171670c-a086-4021-b7c3-ae496b5ab855 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:14.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:29:14.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:29:16.578Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:29:17.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b21379c-fe5e-43aa-8140-006c1656d004 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:17.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b21379c-fe5e-43aa-8140-006c1656d004 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f0ce708 2021-05-28T00:29:17.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b21379c-fe5e-43aa-8140-006c1656d004 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:17.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b21379c-fe5e-43aa-8140-006c1656d004 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:17.762Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2b21379c-fe5e-43aa-8140-006c1656d004 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:17.762Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2b21379c-fe5e-43aa-8140-006c1656d004 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:17.963Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=db924359-5a05-4d88-ab62-db1dd7507d17 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:29:17.963422Z InvocationID=b5320b0f-73e4-4eca-a4d9-7432646e3894 LogTimestamp=2021-05-28T00:29:17.963485Z Entering 2021-05-28T00:29:17.964Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=db924359-5a05-4d88-ab62-db1dd7507d17 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:29:17.963422Z InvocationID=b5320b0f-73e4-4eca-a4d9-7432646e3894 LogTimestamp=2021-05-28T00:29:17.963485Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:29:18.064Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:29:17.964Z InvocationID=77dbac96-72d1-403b-ba50-7aebc2b4f035 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:29:18.072Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:29:17.964Z InvocationID=77dbac96-72d1-403b-ba50-7aebc2b4f035 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:29:18.072Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:29:17.964Z InvocationID=77dbac96-72d1-403b-ba50-7aebc2b4f035 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:29:18.073Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=109 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:29:17.964Z InvocationID=77dbac96-72d1-403b-ba50-7aebc2b4f035 ErrorCode=500 LogTimestamp=2021-05-28T00:29:18.073168Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:29:18.073Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:29:18.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b58e975-c2eb-4794-ad11-510f68a35819 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:18.487Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b58e975-c2eb-4794-ad11-510f68a35819 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ce35b16 2021-05-28T00:29:18.487Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b58e975-c2eb-4794-ad11-510f68a35819 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:18.487Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b58e975-c2eb-4794-ad11-510f68a35819 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:29:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:29:20.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dff32e4-f24e-4839-8838-4659f8473383 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:20.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dff32e4-f24e-4839-8838-4659f8473383 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52431854 2021-05-28T00:29:20.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dff32e4-f24e-4839-8838-4659f8473383 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:20.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dff32e4-f24e-4839-8838-4659f8473383 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:22.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:22.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a7f81b3 2021-05-28T00:29:22.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:22.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:25.763Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:29:25.764Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:29:25.764Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:25.764Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9a03fd2b-f19b-4f9a-9037-8037eaafdf74 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2101714f-b4f7-4dfe-ae0e-57981c6c24f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2101714f-b4f7-4dfe-ae0e-57981c6c24f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25508765 2021-05-28T00:29:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2101714f-b4f7-4dfe-ae0e-57981c6c24f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2101714f-b4f7-4dfe-ae0e-57981c6c24f3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:29:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:29:30.077Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2101714f-b4f7-4dfe-ae0e-57981c6c24f3 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:30.077Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2101714f-b4f7-4dfe-ae0e-57981c6c24f3 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:30.361Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0ce0dee0-1812-43aa-85a5-a47dd3b193c0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:29:30.361329Z InvocationID=a2a1549d-0e5e-4878-b81d-92f40411a3df LogTimestamp=2021-05-28T00:29:30.361450Z Entering 2021-05-28T00:29:30.361Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0ce0dee0-1812-43aa-85a5-a47dd3b193c0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.177 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.177 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:29:30.361329Z InvocationID=a2a1549d-0e5e-4878-b81d-92f40411a3df LogTimestamp=2021-05-28T00:29:30.361450Z localAddr=10.242.102.221 remoteAddr=10.242.170.177 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:29:30.362Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:29:30.362Z InvocationID=378eb0ba-11d6-4fc7-a4c0-a6e18804bb2a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:29:30.362Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:29:30.362Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:29:30.362Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=378eb0ba-11d6-4fc7-a4c0-a6e18804bb2a ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@2ecc95dd 2021-05-28T00:29:30.362Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:29:30.362Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.177 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:29:30.362Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=378eb0ba-11d6-4fc7-a4c0-a6e18804bb2a ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@2ecc95dd 2021-05-28T00:29:30.363Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:29:30.363012Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:29:30.363Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:29:30.363012Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:29:30.363Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:29:31.679Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:29:32.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b9bffad-256b-4415-8f45-e1e348147389 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:32.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b9bffad-256b-4415-8f45-e1e348147389 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79b66cb1 2021-05-28T00:29:32.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b9bffad-256b-4415-8f45-e1e348147389 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:32.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b9bffad-256b-4415-8f45-e1e348147389 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:29:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:29:35.570Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7b9bffad-256b-4415-8f45-e1e348147389 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:29:35.570Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7b9bffad-256b-4415-8f45-e1e348147389 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:29:35.571Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7b9bffad-256b-4415-8f45-e1e348147389 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:35.571Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=7b9bffad-256b-4415-8f45-e1e348147389 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:29:36.556Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe6d804d-2bfb-4e67-9ee0-917726320069 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:36.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe6d804d-2bfb-4e67-9ee0-917726320069 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f62182d 2021-05-28T00:29:36.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe6d804d-2bfb-4e67-9ee0-917726320069 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:36.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe6d804d-2bfb-4e67-9ee0-917726320069 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:36.877Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fe6d804d-2bfb-4e67-9ee0-917726320069 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:36.877Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fe6d804d-2bfb-4e67-9ee0-917726320069 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:29:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:29:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:29:37.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c365849e-981e-4392-bd76-3414f7ecaea1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:37.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c365849e-981e-4392-bd76-3414f7ecaea1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27156f2f 2021-05-28T00:29:37.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c365849e-981e-4392-bd76-3414f7ecaea1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:37.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c365849e-981e-4392-bd76-3414f7ecaea1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:39.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cb23e15-aebc-4895-bc8b-cded458b20af PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:39.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cb23e15-aebc-4895-bc8b-cded458b20af PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d8c5c98 2021-05-28T00:29:39.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cb23e15-aebc-4895-bc8b-cded458b20af PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:39.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cb23e15-aebc-4895-bc8b-cded458b20af PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:40.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:40.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22608274 2021-05-28T00:29:40.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:40.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:29:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:29:43.464Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:29:43.464Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:29:43.464Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:43.464Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=326f9fb2-7121-4d8d-b533-014bbd9c8cde ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:43.568Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:29:44.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40933b17-5a50-43cd-967c-a22a10cef36d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:44.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40933b17-5a50-43cd-967c-a22a10cef36d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ac899bc 2021-05-28T00:29:44.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40933b17-5a50-43cd-967c-a22a10cef36d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:44.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40933b17-5a50-43cd-967c-a22a10cef36d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:45.275Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=40933b17-5a50-43cd-967c-a22a10cef36d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:45.275Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=40933b17-5a50-43cd-967c-a22a10cef36d ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:29:46.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a913130-8f0b-4f72-b168-1ae02ea72e34 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:46.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a913130-8f0b-4f72-b168-1ae02ea72e34 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@194f071c 2021-05-28T00:29:46.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a913130-8f0b-4f72-b168-1ae02ea72e34 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:46.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a913130-8f0b-4f72-b168-1ae02ea72e34 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:47.311Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7a913130-8f0b-4f72-b168-1ae02ea72e34 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:47.311Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7a913130-8f0b-4f72-b168-1ae02ea72e34 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:49.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11332fa3-7f1d-422a-8056-cecf2764a1b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:49.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11332fa3-7f1d-422a-8056-cecf2764a1b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f1a4563 2021-05-28T00:29:49.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11332fa3-7f1d-422a-8056-cecf2764a1b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:49.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11332fa3-7f1d-422a-8056-cecf2764a1b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:29:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:29:51.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:29:51.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61412e45 2021-05-28T00:29:51.191Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:29:51.191Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:29:52.130Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:29:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:29:53.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:29:54.197Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:29:54.197Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:29:54.197Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:54.197Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4a9662f4-fc5f-40f4-9bd6-95c030b518a6 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:29:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:29:59.259Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:29:59.275Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:29:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:30:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:30:05.267Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:30:05.268Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:30:05.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:30:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:30:05.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:30:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:30:11.567Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:30:11.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:11.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30b61db9 2021-05-28T00:30:11.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:11.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:30:14.907Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:30:14.907Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:30:14.908Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:14.908Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=711ecafc-0a5a-4354-b89b-34d9121069e7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:30:16.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22505d4-4723-48f9-add8-b9fcd58db7a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:16.539Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22505d4-4723-48f9-add8-b9fcd58db7a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c7cfd4e 2021-05-28T00:30:16.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22505d4-4723-48f9-add8-b9fcd58db7a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:16.540Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a22505d4-4723-48f9-add8-b9fcd58db7a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:30:17.162Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a22505d4-4723-48f9-add8-b9fcd58db7a1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:17.163Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a22505d4-4723-48f9-add8-b9fcd58db7a1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:18.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:18.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@547c5b7a 2021-05-28T00:30:18.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:18.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:30:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:30:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:30:21.079Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:30:21.080Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:30:21.080Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:21.081Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6fd2c4d4-1144-47c4-9156-46aa92340155 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:30:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:30:24.184Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f3f09f-9d87-4e57-ae3a-530b1abaa140 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:24.184Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f3f09f-9d87-4e57-ae3a-530b1abaa140 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d3108a 2021-05-28T00:30:24.184Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f3f09f-9d87-4e57-ae3a-530b1abaa140 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:24.184Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f3f09f-9d87-4e57-ae3a-530b1abaa140 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:30:25.063Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=38f3f09f-9d87-4e57-ae3a-530b1abaa140 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:25.063Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=38f3f09f-9d87-4e57-ae3a-530b1abaa140 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:25.686Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:30:25.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:25.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@302f947d 2021-05-28T00:30:25.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:25.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:30:28.959Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:30:28.959Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:30:28.960Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:28.960Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=75cb5116-7fa5-40e3-b9f7-c2cb3fe40056 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:30:31.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b484e7-76af-4e1e-9da8-0193a2cc4bba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:31.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b484e7-76af-4e1e-9da8-0193a2cc4bba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bebe89 2021-05-28T00:30:31.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b484e7-76af-4e1e-9da8-0193a2cc4bba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:31.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b484e7-76af-4e1e-9da8-0193a2cc4bba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:30:32.376Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f7b484e7-76af-4e1e-9da8-0193a2cc4bba ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:32.376Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f7b484e7-76af-4e1e-9da8-0193a2cc4bba ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:30:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:30:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:30:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:30:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:30:36.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:36.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@359f6c9a 2021-05-28T00:30:36.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:36.433Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:30:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:30:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:30:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:30:39.440Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:30:39.440Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:30:39.440Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:39.440Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0f04ad5a-5aef-4d6f-9151-292673ed5e4f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:30:44.463Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:30:44.469Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:30:49.578Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:30:50.468Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:30:50.469Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:30:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:30:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:30:52.145Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:30:57.476Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:30:57.477Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:30:59.522Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:30:59.570Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:30:59.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:30:59.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@795f9a28 2021-05-28T00:30:59.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:30:59.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:01.557Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:31:01.558Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 8000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:02.603Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:31:02.603Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:31:02.603Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:31:02.604Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=f1c6fb45-bab7-4114-8fdc-aeb551ebad22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:31:02.670Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:02.674Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:02.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:02.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10c00dbd 2021-05-28T00:31:02.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:02.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:31:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:31:05.699Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:31:05.700Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:31:05.700Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:31:05.700Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=6bc795c4-9748-4a57-816b-a393c97a2fa0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:31:05.704Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:05.706Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:05.706Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8dc2463-9a88-4711-b1f3-b05b48628e7d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:05.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8dc2463-9a88-4711-b1f3-b05b48628e7d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5bfe07af 2021-05-28T00:31:05.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8dc2463-9a88-4711-b1f3-b05b48628e7d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:05.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8dc2463-9a88-4711-b1f3-b05b48628e7d PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:08.735Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e8dc2463-9a88-4711-b1f3-b05b48628e7d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:31:08.735Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e8dc2463-9a88-4711-b1f3-b05b48628e7d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:31:08.771Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:08.775Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:08.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbe3a199-6e65-4424-a8df-5d149b7d9c6f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:08.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbe3a199-6e65-4424-a8df-5d149b7d9c6f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70d53c20 2021-05-28T00:31:08.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbe3a199-6e65-4424-a8df-5d149b7d9c6f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:08.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbe3a199-6e65-4424-a8df-5d149b7d9c6f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:09.571Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:31:09.572Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 16000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:11.803Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=dbe3a199-6e65-4424-a8df-5d149b7d9c6f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:31:11.803Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=dbe3a199-6e65-4424-a8df-5d149b7d9c6f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:31:11.868Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:11.872Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:11.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=749a6423-55e3-4126-99e9-07f801815f95 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:11.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=749a6423-55e3-4126-99e9-07f801815f95 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ff8efab 2021-05-28T00:31:11.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=749a6423-55e3-4126-99e9-07f801815f95 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:11.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=749a6423-55e3-4126-99e9-07f801815f95 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:14.899Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=749a6423-55e3-4126-99e9-07f801815f95 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:31:14.899Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=749a6423-55e3-4126-99e9-07f801815f95 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:31:14.970Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:14.973Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:14.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=679d6d37-edf0-4fef-bb01-336d7c29deaf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:14.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=679d6d37-edf0-4fef-bb01-336d7c29deaf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@559d98e3 2021-05-28T00:31:14.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=679d6d37-edf0-4fef-bb01-336d7c29deaf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:14.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=679d6d37-edf0-4fef-bb01-336d7c29deaf PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:15.674Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:15.674Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:31:16.674Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:17.675Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:17.998Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=679d6d37-edf0-4fef-bb01-336d7c29deaf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:31:17.999Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=679d6d37-edf0-4fef-bb01-336d7c29deaf ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:31:18.068Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:18.071Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:18.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:18.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bf9ae5c 2021-05-28T00:31:18.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:18.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:18.675Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:19.675Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:31:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:31:20.675Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:21.097Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:31:21.098Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:31:21.098Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:31:21.098Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=759fddbf-ab31-48c4-94fb-5d9e49c49ef1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:31:21.168Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:21.676Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:22.676Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:23.676Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:25.657Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:31:25.658Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 32000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:26.474Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:31:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:31:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:31:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:31:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:31:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:41.223Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.223Z [cluster1-timeouter-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.140.16:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.271Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.271Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d4826a0-23a8-447c-874a-35ab81a6338f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.271Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d4826a0-23a8-447c-874a-35ab81a6338f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ae987d9 2021-05-28T00:31:41.272Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d4826a0-23a8-447c-874a-35ab81a6338f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.272Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d4826a0-23a8-447c-874a-35ab81a6338f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.278Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8d4826a0-23a8-447c-874a-35ab81a6338f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.278Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8d4826a0-23a8-447c-874a-35ab81a6338f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.282Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.369Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7824ced-009c-4354-8aba-a447cb0390d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7824ced-009c-4354-8aba-a447cb0390d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b46ff15 2021-05-28T00:31:41.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7824ced-009c-4354-8aba-a447cb0390d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7824ced-009c-4354-8aba-a447cb0390d3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.372Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a7824ced-009c-4354-8aba-a447cb0390d3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.373Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a7824ced-009c-4354-8aba-a447cb0390d3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.376Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.379Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.380Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7867d8bd-b607-4352-8252-c4ba0a8eed22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.380Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7867d8bd-b607-4352-8252-c4ba0a8eed22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c62bcfd 2021-05-28T00:31:41.380Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7867d8bd-b607-4352-8252-c4ba0a8eed22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.380Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7867d8bd-b607-4352-8252-c4ba0a8eed22 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.382Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7867d8bd-b607-4352-8252-c4ba0a8eed22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.382Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7867d8bd-b607-4352-8252-c4ba0a8eed22 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:41.474Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.478Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2461a922-2ec3-4933-a59c-e49c4b0c29b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2461a922-2ec3-4933-a59c-e49c4b0c29b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3452e2ac 2021-05-28T00:31:41.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2461a922-2ec3-4933-a59c-e49c4b0c29b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2461a922-2ec3-4933-a59c-e49c4b0c29b0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.480Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2461a922-2ec3-4933-a59c-e49c4b0c29b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.481Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2461a922-2ec3-4933-a59c-e49c4b0c29b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.568Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.573Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6e290e9-d291-4a54-b690-5d4e89b45fb0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6e290e9-d291-4a54-b690-5d4e89b45fb0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ea716d8 2021-05-28T00:31:41.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6e290e9-d291-4a54-b690-5d4e89b45fb0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6e290e9-d291-4a54-b690-5d4e89b45fb0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.575Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e6e290e9-d291-4a54-b690-5d4e89b45fb0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.575Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e6e290e9-d291-4a54-b690-5d4e89b45fb0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.578Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.671Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ae577d4-46f1-4bd1-8b4b-b0f303586013 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ae577d4-46f1-4bd1-8b4b-b0f303586013 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55cc6f39 2021-05-28T00:31:41.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ae577d4-46f1-4bd1-8b4b-b0f303586013 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ae577d4-46f1-4bd1-8b4b-b0f303586013 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.673Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=1ae577d4-46f1-4bd1-8b4b-b0f303586013 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.674Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=1ae577d4-46f1-4bd1-8b4b-b0f303586013 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.678Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.681Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c599ac5-68fc-4e47-bb88-d0cc1a29fd3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c599ac5-68fc-4e47-bb88-d0cc1a29fd3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ae503d5 2021-05-28T00:31:41.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c599ac5-68fc-4e47-bb88-d0cc1a29fd3f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c599ac5-68fc-4e47-bb88-d0cc1a29fd3f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.682Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3c599ac5-68fc-4e47-bb88-d0cc1a29fd3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.683Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3c599ac5-68fc-4e47-bb88-d0cc1a29fd3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.768Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.785Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ce84f4a-17c6-44d9-b570-5289020fcd34 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ce84f4a-17c6-44d9-b570-5289020fcd34 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f5badda 2021-05-28T00:31:41.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ce84f4a-17c6-44d9-b570-5289020fcd34 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ce84f4a-17c6-44d9-b570-5289020fcd34 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.787Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3ce84f4a-17c6-44d9-b570-5289020fcd34 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.787Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3ce84f4a-17c6-44d9-b570-5289020fcd34 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.790Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.868Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d60206-3445-4434-8916-933ab16911f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d60206-3445-4434-8916-933ab16911f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77809b8c 2021-05-28T00:31:41.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d60206-3445-4434-8916-933ab16911f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d60206-3445-4434-8916-933ab16911f3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.870Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=84d60206-3445-4434-8916-933ab16911f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.871Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=84d60206-3445-4434-8916-933ab16911f3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.874Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.876Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db20528-ab9c-47b6-bbea-ddcea3fcfa87 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db20528-ab9c-47b6-bbea-ddcea3fcfa87 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59809788 2021-05-28T00:31:41.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db20528-ab9c-47b6-bbea-ddcea3fcfa87 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db20528-ab9c-47b6-bbea-ddcea3fcfa87 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.879Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2db20528-ab9c-47b6-bbea-ddcea3fcfa87 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.879Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2db20528-ab9c-47b6-bbea-ddcea3fcfa87 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.882Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.885Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05285ce4-17de-45f4-8fd2-b3a2fa1083ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05285ce4-17de-45f4-8fd2-b3a2fa1083ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31e7f946 2021-05-28T00:31:41.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05285ce4-17de-45f4-8fd2-b3a2fa1083ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05285ce4-17de-45f4-8fd2-b3a2fa1083ad PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.886Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=05285ce4-17de-45f4-8fd2-b3a2fa1083ad ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.887Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=05285ce4-17de-45f4-8fd2-b3a2fa1083ad ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.890Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.893Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.893Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d1abc72-d0ce-43a5-887a-231b6e8580e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.893Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d1abc72-d0ce-43a5-887a-231b6e8580e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5926c738 2021-05-28T00:31:41.893Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d1abc72-d0ce-43a5-887a-231b6e8580e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.893Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d1abc72-d0ce-43a5-887a-231b6e8580e9 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.895Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3d1abc72-d0ce-43a5-887a-231b6e8580e9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.895Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3d1abc72-d0ce-43a5-887a-231b6e8580e9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.898Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.969Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d68b49e-8aca-4efa-b5d6-40d6524c848e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d68b49e-8aca-4efa-b5d6-40d6524c848e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@577e4ec6 2021-05-28T00:31:41.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d68b49e-8aca-4efa-b5d6-40d6524c848e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d68b49e-8aca-4efa-b5d6-40d6524c848e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.971Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7d68b49e-8aca-4efa-b5d6-40d6524c848e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.971Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7d68b49e-8aca-4efa-b5d6-40d6524c848e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.974Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.977Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06dafb9f-a883-4605-aa39-9ae0ccacd7fb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06dafb9f-a883-4605-aa39-9ae0ccacd7fb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@593a5f1 2021-05-28T00:31:41.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06dafb9f-a883-4605-aa39-9ae0ccacd7fb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06dafb9f-a883-4605-aa39-9ae0ccacd7fb PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.979Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=06dafb9f-a883-4605-aa39-9ae0ccacd7fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.979Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=06dafb9f-a883-4605-aa39-9ae0ccacd7fb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.982Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.984Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:41.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fbb2b1-2b6a-4e2f-ae69-6a4529821962 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:41.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fbb2b1-2b6a-4e2f-ae69-6a4529821962 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@145ca216 2021-05-28T00:31:41.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fbb2b1-2b6a-4e2f-ae69-6a4529821962 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:41.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fbb2b1-2b6a-4e2f-ae69-6a4529821962 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:41.986Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=12fbb2b1-2b6a-4e2f-ae69-6a4529821962 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:41.986Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=12fbb2b1-2b6a-4e2f-ae69-6a4529821962 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:41.989Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.068Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ec88853-ec98-4291-81df-5c7445e26ec7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ec88853-ec98-4291-81df-5c7445e26ec7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b687a1a 2021-05-28T00:31:42.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ec88853-ec98-4291-81df-5c7445e26ec7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ec88853-ec98-4291-81df-5c7445e26ec7 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.071Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3ec88853-ec98-4291-81df-5c7445e26ec7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.071Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3ec88853-ec98-4291-81df-5c7445e26ec7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.074Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.076Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4ba94c-4506-4dca-a2a9-35b52629dcb2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4ba94c-4506-4dca-a2a9-35b52629dcb2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@350f84b7 2021-05-28T00:31:42.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4ba94c-4506-4dca-a2a9-35b52629dcb2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff4ba94c-4506-4dca-a2a9-35b52629dcb2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.078Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ff4ba94c-4506-4dca-a2a9-35b52629dcb2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.079Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ff4ba94c-4506-4dca-a2a9-35b52629dcb2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.081Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.084Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20706e27-adb7-4249-9b98-f69be083d800 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20706e27-adb7-4249-9b98-f69be083d800 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9faf829 2021-05-28T00:31:42.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20706e27-adb7-4249-9b98-f69be083d800 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20706e27-adb7-4249-9b98-f69be083d800 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.086Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=20706e27-adb7-4249-9b98-f69be083d800 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.086Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=20706e27-adb7-4249-9b98-f69be083d800 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.172Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.180Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a4884a9-f542-442d-8a28-6637701ec7b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a4884a9-f542-442d-8a28-6637701ec7b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1268adb3 2021-05-28T00:31:42.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a4884a9-f542-442d-8a28-6637701ec7b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a4884a9-f542-442d-8a28-6637701ec7b2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.182Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3a4884a9-f542-442d-8a28-6637701ec7b2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.182Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3a4884a9-f542-442d-8a28-6637701ec7b2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.187Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.269Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9dc29e-038a-49f8-9da1-2aab754086c1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9dc29e-038a-49f8-9da1-2aab754086c1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a74799f 2021-05-28T00:31:42.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9dc29e-038a-49f8-9da1-2aab754086c1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9dc29e-038a-49f8-9da1-2aab754086c1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.271Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6c9dc29e-038a-49f8-9da1-2aab754086c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.271Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6c9dc29e-038a-49f8-9da1-2aab754086c1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.275Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.278Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79e7e5be-9a08-4d3a-b6f1-697a3d40d525 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79e7e5be-9a08-4d3a-b6f1-697a3d40d525 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b293879 2021-05-28T00:31:42.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79e7e5be-9a08-4d3a-b6f1-697a3d40d525 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79e7e5be-9a08-4d3a-b6f1-697a3d40d525 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.280Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=79e7e5be-9a08-4d3a-b6f1-697a3d40d525 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.280Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=79e7e5be-9a08-4d3a-b6f1-697a3d40d525 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.367Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.372Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5e9a7d9-cc2e-4fa1-83c1-b0b762066a53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5e9a7d9-cc2e-4fa1-83c1-b0b762066a53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@996b74e 2021-05-28T00:31:42.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5e9a7d9-cc2e-4fa1-83c1-b0b762066a53 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5e9a7d9-cc2e-4fa1-83c1-b0b762066a53 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.375Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b5e9a7d9-cc2e-4fa1-83c1-b0b762066a53 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.375Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b5e9a7d9-cc2e-4fa1-83c1-b0b762066a53 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.380Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.384Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.384Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5de7e7ce-deec-4c09-b574-6de8baeb98b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.384Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5de7e7ce-deec-4c09-b574-6de8baeb98b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4faa2ea6 2021-05-28T00:31:42.384Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5de7e7ce-deec-4c09-b574-6de8baeb98b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.384Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5de7e7ce-deec-4c09-b574-6de8baeb98b0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.386Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5de7e7ce-deec-4c09-b574-6de8baeb98b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.386Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5de7e7ce-deec-4c09-b574-6de8baeb98b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:42.468Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.472Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:42.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9337db13-4a17-4533-82e8-8706c03dbe7a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:42.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9337db13-4a17-4533-82e8-8706c03dbe7a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f53b9d8 2021-05-28T00:31:42.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9337db13-4a17-4533-82e8-8706c03dbe7a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:42.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9337db13-4a17-4533-82e8-8706c03dbe7a PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:42.474Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9337db13-4a17-4533-82e8-8706c03dbe7a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:42.475Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9337db13-4a17-4533-82e8-8706c03dbe7a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:43.068Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:43.074Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:43.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9e896a-0096-47d4-8a12-4321076f4c97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:43.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9e896a-0096-47d4-8a12-4321076f4c97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d50784a 2021-05-28T00:31:43.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9e896a-0096-47d4-8a12-4321076f4c97 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:43.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9e896a-0096-47d4-8a12-4321076f4c97 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:43.076Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ce9e896a-0096-47d4-8a12-4321076f4c97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:31:43.076Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ce9e896a-0096-47d4-8a12-4321076f4c97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:46.052Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:46.054Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:46.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc38366e-5f37-493f-a693-00177d5c7c43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:46.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc38366e-5f37-493f-a693-00177d5c7c43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65427ef 2021-05-28T00:31:46.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc38366e-5f37-493f-a693-00177d5c7c43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:46.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc38366e-5f37-493f-a693-00177d5c7c43 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:49.066Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=bc38366e-5f37-493f-a693-00177d5c7c43 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:31:49.066Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=bc38366e-5f37-493f-a693-00177d5c7c43 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:49.072Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:49.076Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:49.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ffee658-2980-483f-8d78-b734acc9b1f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:49.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ffee658-2980-483f-8d78-b734acc9b1f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16ab48ea 2021-05-28T00:31:49.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ffee658-2980-483f-8d78-b734acc9b1f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:49.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ffee658-2980-483f-8d78-b734acc9b1f6 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:31:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:31:52.087Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7ffee658-2980-483f-8d78-b734acc9b1f6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:31:52.088Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7ffee658-2980-483f-8d78-b734acc9b1f6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:52.143Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:31:52.172Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:52.175Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:52.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b5b594f-613d-417c-85c7-6cbb2e7aaf97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:52.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b5b594f-613d-417c-85c7-6cbb2e7aaf97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b38996d 2021-05-28T00:31:52.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b5b594f-613d-417c-85c7-6cbb2e7aaf97 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:52.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b5b594f-613d-417c-85c7-6cbb2e7aaf97 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:55.187Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4b5b594f-613d-417c-85c7-6cbb2e7aaf97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:31:55.187Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4b5b594f-613d-417c-85c7-6cbb2e7aaf97 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:55.191Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:55.194Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:55.194Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=717c2ae9-8f96-420e-ae9f-099c87bfafc4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:55.194Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=717c2ae9-8f96-420e-ae9f-099c87bfafc4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60346355 2021-05-28T00:31:55.194Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=717c2ae9-8f96-420e-ae9f-099c87bfafc4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:55.194Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=717c2ae9-8f96-420e-ae9f-099c87bfafc4 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:31:57.663Z [cluster1-reconnection-1] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:31:57.664Z [cluster1-reconnection-1] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 64000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:58.204Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=717c2ae9-8f96-420e-ae9f-099c87bfafc4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:31:58.204Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=717c2ae9-8f96-420e-ae9f-099c87bfafc4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:31:58.269Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:58.272Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:31:58.272Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52db3c81-6838-4a7d-b70f-48d8e97d285a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:31:58.272Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52db3c81-6838-4a7d-b70f-48d8e97d285a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2607c68b 2021-05-28T00:31:58.272Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52db3c81-6838-4a7d-b70f-48d8e97d285a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:31:58.272Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52db3c81-6838-4a7d-b70f-48d8e97d285a PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:31:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:01.279Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=52db3c81-6838-4a7d-b70f-48d8e97d285a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:01.279Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=52db3c81-6838-4a7d-b70f-48d8e97d285a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:01.369Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:01.373Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:01.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2353bbe3-0ed2-4050-a655-78606efd0dc6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:01.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2353bbe3-0ed2-4050-a655-78606efd0dc6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6de78d1b 2021-05-28T00:32:01.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2353bbe3-0ed2-4050-a655-78606efd0dc6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:01.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2353bbe3-0ed2-4050-a655-78606efd0dc6 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:04.381Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2353bbe3-0ed2-4050-a655-78606efd0dc6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:04.381Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2353bbe3-0ed2-4050-a655-78606efd0dc6 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:04.386Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:04.469Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:04.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f48d80c-5c8a-4ce5-abff-a632688ca7d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:04.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f48d80c-5c8a-4ce5-abff-a632688ca7d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72b9ad39 2021-05-28T00:32:04.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f48d80c-5c8a-4ce5-abff-a632688ca7d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:04.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f48d80c-5c8a-4ce5-abff-a632688ca7d1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:05.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:32:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:32:07.479Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3f48d80c-5c8a-4ce5-abff-a632688ca7d1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:07.479Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3f48d80c-5c8a-4ce5-abff-a632688ca7d1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:07.484Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:07.569Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=220b5005-d6eb-4862-9318-fcc5a1914045 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=220b5005-d6eb-4862-9318-fcc5a1914045 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e7ad54d 2021-05-28T00:32:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=220b5005-d6eb-4862-9318-fcc5a1914045 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=220b5005-d6eb-4862-9318-fcc5a1914045 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:10.577Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=220b5005-d6eb-4862-9318-fcc5a1914045 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:10.577Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=220b5005-d6eb-4862-9318-fcc5a1914045 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:10.581Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:10.668Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:10.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a0e0d1-e7ab-4752-a4a2-b78c9adb9e1b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:10.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a0e0d1-e7ab-4752-a4a2-b78c9adb9e1b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b167b52 2021-05-28T00:32:10.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a0e0d1-e7ab-4752-a4a2-b78c9adb9e1b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:10.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a0e0d1-e7ab-4752-a4a2-b78c9adb9e1b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:13.677Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=41a0e0d1-e7ab-4752-a4a2-b78c9adb9e1b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:13.678Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=41a0e0d1-e7ab-4752-a4a2-b78c9adb9e1b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:13.682Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:13.685Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:13.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73b2988a-cccb-4c9d-91a3-4b2cfdf498f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:13.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73b2988a-cccb-4c9d-91a3-4b2cfdf498f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cb2769c 2021-05-28T00:32:13.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73b2988a-cccb-4c9d-91a3-4b2cfdf498f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:13.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73b2988a-cccb-4c9d-91a3-4b2cfdf498f0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:14.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:16.699Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=73b2988a-cccb-4c9d-91a3-4b2cfdf498f0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:16.704Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=73b2988a-cccb-4c9d-91a3-4b2cfdf498f0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:16.708Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:16.711Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:16.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e83847bb-2cd4-46c9-8ed3-df6d9737f89b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:16.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e83847bb-2cd4-46c9-8ed3-df6d9737f89b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e91a3be 2021-05-28T00:32:16.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e83847bb-2cd4-46c9-8ed3-df6d9737f89b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:16.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e83847bb-2cd4-46c9-8ed3-df6d9737f89b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:17.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:18.381Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3b0eede9-d86d-4a2c-9d66-c38004b05c14 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:32:18.380317Z InvocationID=95788346-eab8-4773-883d-bf2f9efd3244 LogTimestamp=2021-05-28T00:32:18.380457Z Entering 2021-05-28T00:32:18.381Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3b0eede9-d86d-4a2c-9d66-c38004b05c14 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:32:18.380317Z InvocationID=95788346-eab8-4773-883d-bf2f9efd3244 LogTimestamp=2021-05-28T00:32:18.380457Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:32:18.382Z [qtp580718781-12] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:32:18.381Z InvocationID=1d6df9df-40d9-4711-8a1f-45caa7b8e384 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:32:18.383Z [qtp580718781-12] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=2 EndTimestamp=2021-05-28T00:32:18.383Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:32:18.381Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=1d6df9df-40d9-4711-8a1f-45caa7b8e384 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@4bb90907 2021-05-28T00:32:18.383Z [qtp580718781-12] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=2 EndTimestamp=2021-05-28T00:32:18.383Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:32:18.381Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=1d6df9df-40d9-4711-8a1f-45caa7b8e384 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@4bb90907 2021-05-28T00:32:18.383Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:32:18.383809Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:32:18.384Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:32:18.383809Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:32:18.384Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:32:19.720Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e83847bb-2cd4-46c9-8ed3-df6d9737f89b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:19.720Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e83847bb-2cd4-46c9-8ed3-df6d9737f89b ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:19.770Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:19.774Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:19.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62361507-0e66-458a-8922-2ab3c4321874 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:19.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62361507-0e66-458a-8922-2ab3c4321874 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35d9afdc 2021-05-28T00:32:19.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62361507-0e66-458a-8922-2ab3c4321874 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:19.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62361507-0e66-458a-8922-2ab3c4321874 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:20.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:32:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:32:22.786Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=62361507-0e66-458a-8922-2ab3c4321874 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:22.787Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=62361507-0e66-458a-8922-2ab3c4321874 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:22.870Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:22.873Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:22.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c185eb9-219d-4263-b0ee-e5f96ae82c02 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:22.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c185eb9-219d-4263-b0ee-e5f96ae82c02 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cdf2af1 2021-05-28T00:32:22.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c185eb9-219d-4263-b0ee-e5f96ae82c02 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:22.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c185eb9-219d-4263-b0ee-e5f96ae82c02 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:24.305Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator. (SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator. (ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:84) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:80) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:32:25.884Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4c185eb9-219d-4263-b0ee-e5f96ae82c02 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:32:25.884Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4c185eb9-219d-4263-b0ee-e5f96ae82c02 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:32:25.889Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:29.476Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:32:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:32:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:32:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:32:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:32:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:45.215Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator. (SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator. (ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:84) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:80) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2021-05-28T00:32:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:32:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:32:52.126Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:32:52.677Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=615b94ae-e84c-4c63-803c-daf2a5c79560 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:32:52.676703Z InvocationID=86414619-1b0a-48b5-b89f-06ca6eaffcee LogTimestamp=2021-05-28T00:32:52.676856Z Entering 2021-05-28T00:32:52.677Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=615b94ae-e84c-4c63-803c-daf2a5c79560 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:32:52.676703Z InvocationID=86414619-1b0a-48b5-b89f-06ca6eaffcee LogTimestamp=2021-05-28T00:32:52.676856Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:32:52.678Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:32:52.677Z InvocationID=94ba33b5-fed8-404c-a040-2af1176188bc ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:32:52.679Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:32:52.678Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:32:52.677Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=94ba33b5-fed8-404c-a040-2af1176188bc ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@4d1a2fb2 2021-05-28T00:32:52.679Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:32:52.678Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:32:52.677Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=94ba33b5-fed8-404c-a040-2af1176188bc ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@4d1a2fb2 2021-05-28T00:32:52.679Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:32:52.679488Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:32:52.679Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:32:52.679488Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:32:52.680Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:32:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:32:58.222Z [cluster1-timeouter-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.OperationTimedOutException: [cassandra.onap/10.242.140.16:9042] Operation timed out at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.222Z [cluster1-timeouter-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.140.16:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.140.16:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onTimeout(Connection.java:1526) at com.datastax.driver.core.Connection$ResponseHandler$1.run(Connection.java:1605) at io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:672) at io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:747) at io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:472) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.273Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87156e24-f90b-45c6-9ead-3e764fb327f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87156e24-f90b-45c6-9ead-3e764fb327f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f0fb967 2021-05-28T00:32:58.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87156e24-f90b-45c6-9ead-3e764fb327f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87156e24-f90b-45c6-9ead-3e764fb327f7 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.369Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.373Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=705ac75c-a0e8-4453-82a6-c9172353ee46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=705ac75c-a0e8-4453-82a6-c9172353ee46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@524de171 2021-05-28T00:32:58.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=705ac75c-a0e8-4453-82a6-c9172353ee46 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=705ac75c-a0e8-4453-82a6-c9172353ee46 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.385Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.469Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=823c357e-6631-4bd4-a770-6c698573584d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=823c357e-6631-4bd4-a770-6c698573584d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77903603 2021-05-28T00:32:58.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=823c357e-6631-4bd4-a770-6c698573584d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=823c357e-6631-4bd4-a770-6c698573584d PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.476Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.478Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b92eb19-d6c0-416f-b791-4915e7232085 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b92eb19-d6c0-416f-b791-4915e7232085 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55f66d17 2021-05-28T00:32:58.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b92eb19-d6c0-416f-b791-4915e7232085 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b92eb19-d6c0-416f-b791-4915e7232085 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.484Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.570Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6954ce2d-fc23-4bd5-b8bf-b4b15bcda31a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6954ce2d-fc23-4bd5-b8bf-b4b15bcda31a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44a9f0b2 2021-05-28T00:32:58.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6954ce2d-fc23-4bd5-b8bf-b4b15bcda31a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6954ce2d-fc23-4bd5-b8bf-b4b15bcda31a PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.577Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.580Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02646588-b079-4c70-aadb-d72c7db7d2da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02646588-b079-4c70-aadb-d72c7db7d2da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7952d73f 2021-05-28T00:32:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02646588-b079-4c70-aadb-d72c7db7d2da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02646588-b079-4c70-aadb-d72c7db7d2da PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.669Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.672Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21d46de4-decb-4bc3-a1f8-0648e1dc89da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21d46de4-decb-4bc3-a1f8-0648e1dc89da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f04c638 2021-05-28T00:32:58.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21d46de4-decb-4bc3-a1f8-0648e1dc89da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21d46de4-decb-4bc3-a1f8-0648e1dc89da PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.680Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.772Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43e52d75-cece-40e8-a999-039e54c7bb01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43e52d75-cece-40e8-a999-039e54c7bb01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@697c9c1a 2021-05-28T00:32:58.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43e52d75-cece-40e8-a999-039e54c7bb01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43e52d75-cece-40e8-a999-039e54c7bb01 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.878Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.881Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d252c48f-5eb9-49f4-b0a8-3eac5af5c83e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d252c48f-5eb9-49f4-b0a8-3eac5af5c83e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@625bdde7 2021-05-28T00:32:58.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d252c48f-5eb9-49f4-b0a8-3eac5af5c83e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d252c48f-5eb9-49f4-b0a8-3eac5af5c83e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.968Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.971Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b451c89a-9296-4419-8045-390c9bbc3c70 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b451c89a-9296-4419-8045-390c9bbc3c70 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ddf15bc 2021-05-28T00:32:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b451c89a-9296-4419-8045-390c9bbc3c70 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b451c89a-9296-4419-8045-390c9bbc3c70 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:58.978Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.982Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:58.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726905fe-eeaf-408f-a75d-b6ed190f1627 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:58.983Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726905fe-eeaf-408f-a75d-b6ed190f1627 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24432739 2021-05-28T00:32:58.983Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726905fe-eeaf-408f-a75d-b6ed190f1627 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:58.983Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726905fe-eeaf-408f-a75d-b6ed190f1627 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:59.069Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:59.072Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:32:59.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f29893e-827c-4d86-8641-a2433fb3a3c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:32:59.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f29893e-827c-4d86-8641-a2433fb3a3c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2952e3e7 2021-05-28T00:32:59.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f29893e-827c-4d86-8641-a2433fb3a3c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:32:59.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f29893e-827c-4d86-8641-a2433fb3a3c2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:32:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:01.070Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:01.073Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:01.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dff2800-cf3f-47d4-978b-c57f84a83a22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:01.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dff2800-cf3f-47d4-978b-c57f84a83a22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@155ab7cf 2021-05-28T00:33:01.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dff2800-cf3f-47d4-978b-c57f84a83a22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:01.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7dff2800-cf3f-47d4-978b-c57f84a83a22 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:01.671Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:33:01.672Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 128000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:04.070Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:04.072Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:04.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa583fa4-8801-4a20-aadb-509b5e70d5f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:04.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa583fa4-8801-4a20-aadb-509b5e70d5f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60dd222d 2021-05-28T00:33:04.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa583fa4-8801-4a20-aadb-509b5e70d5f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:04.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa583fa4-8801-4a20-aadb-509b5e70d5f8 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:05.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:33:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:33:07.068Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:07.071Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:07.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e36d5ca8-75fa-428f-8ea4-c4b90588b729 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:07.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e36d5ca8-75fa-428f-8ea4-c4b90588b729 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d1abbe0 2021-05-28T00:33:07.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e36d5ca8-75fa-428f-8ea4-c4b90588b729 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:07.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e36d5ca8-75fa-428f-8ea4-c4b90588b729 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:10.069Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:10.071Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:10.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95524e0b-76bb-40b1-b042-fcdf865d9b19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:10.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95524e0b-76bb-40b1-b042-fcdf865d9b19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63e9367f 2021-05-28T00:33:10.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95524e0b-76bb-40b1-b042-fcdf865d9b19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:10.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95524e0b-76bb-40b1-b042-fcdf865d9b19 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:13.069Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:13.072Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:13.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d12b91c-b4dd-4a0a-a538-3a5304f24b2b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:13.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d12b91c-b4dd-4a0a-a538-3a5304f24b2b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@234d1102 2021-05-28T00:33:13.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d12b91c-b4dd-4a0a-a538-3a5304f24b2b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:13.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d12b91c-b4dd-4a0a-a538-3a5304f24b2b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:14.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:16.052Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:16.054Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:16.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9580d05-4979-4fa4-a751-2c04435af311 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:16.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9580d05-4979-4fa4-a751-2c04435af311 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a08930c 2021-05-28T00:33:16.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9580d05-4979-4fa4-a751-2c04435af311 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:16.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9580d05-4979-4fa4-a751-2c04435af311 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:17.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:19.070Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:19.073Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:19.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb3e1269-f0b3-4de0-8c6d-ace554abcad7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:19.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb3e1269-f0b3-4de0-8c6d-ace554abcad7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22c5f816 2021-05-28T00:33:19.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb3e1269-f0b3-4de0-8c6d-ace554abcad7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:19.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb3e1269-f0b3-4de0-8c6d-ace554abcad7 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:20.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:33:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:33:22.073Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:22.076Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:22.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c72a4c0c-c1da-4a3a-9ea2-5bfa713d4bbf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:22.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c72a4c0c-c1da-4a3a-9ea2-5bfa713d4bbf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8b578ff 2021-05-28T00:33:22.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c72a4c0c-c1da-4a3a-9ea2-5bfa713d4bbf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:22.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c72a4c0c-c1da-4a3a-9ea2-5bfa713d4bbf PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:25.051Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:25.054Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:25.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696aaed0-ff57-42c8-bc90-bd681efefa0e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:25.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696aaed0-ff57-42c8-bc90-bd681efefa0e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@433badea 2021-05-28T00:33:25.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696aaed0-ff57-42c8-bc90-bd681efefa0e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:25.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696aaed0-ff57-42c8-bc90-bd681efefa0e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:26.898Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=29f9f783-4ad0-48bc-ae4f-51e683a34af1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:33:26.897772Z InvocationID=6a432db7-86b5-409b-a02f-bdf57bd82047 LogTimestamp=2021-05-28T00:33:26.897902Z Entering 2021-05-28T00:33:26.898Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=29f9f783-4ad0-48bc-ae4f-51e683a34af1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:33:26.897772Z InvocationID=6a432db7-86b5-409b-a02f-bdf57bd82047 LogTimestamp=2021-05-28T00:33:26.897902Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:33:26.899Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:33:26.898Z InvocationID=1527532e-509c-4ce8-8d76-c111cc79ce6a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:33:26.899Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:33:26.899Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:33:26.898Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=1527532e-509c-4ce8-8d76-c111cc79ce6a ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@66033429 2021-05-28T00:33:26.899Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:33:26.899Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:33:26.898Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=1527532e-509c-4ce8-8d76-c111cc79ce6a ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@66033429 2021-05-28T00:33:26.899Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:33:26.899826Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:33:26.899Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:33:26.899826Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:33:26.900Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:33:28.070Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:28.073Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:28.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fedd87-71ad-4242-acc9-14d45755a779 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:28.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fedd87-71ad-4242-acc9-14d45755a779 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d45d672 2021-05-28T00:33:28.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fedd87-71ad-4242-acc9-14d45755a779 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:28.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44fedd87-71ad-4242-acc9-14d45755a779 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-05-28T00:33:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:29.964Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator. (SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator. (ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:84) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:80) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:33:31.072Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:32.678Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:32.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c88df9c-2780-439b-b688-67f4c36c9fa5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:32.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c88df9c-2780-439b-b688-67f4c36c9fa5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20cd9e84 2021-05-28T00:33:32.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c88df9c-2780-439b-b688-67f4c36c9fa5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:32.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c88df9c-2780-439b-b688-67f4c36c9fa5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:33:32.684Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7c88df9c-2780-439b-b688-67f4c36c9fa5 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:33:32.685Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=7c88df9c-2780-439b-b688-67f4c36c9fa5 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:33:34.069Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:35.480Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [EdgeStoreQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$1.call(BackendTransaction.java:272) at org.janusgraph.diskstorage.BackendTransaction$1.call(BackendTransaction.java:269) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.edgeStoreQuery(BackendTransaction.java:269) at org.janusgraph.graphdb.database.StandardJanusGraph.edgeQuery(StandardJanusGraph.java:436) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.lambda$null$0(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:98) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:90) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:82) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.lambda$getBasicIterator$1(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.vertices.CacheVertex.loadRelations(CacheVertex.java:67) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.getBasicIterator(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.iterator(SimpleVertexQueryProcessor.java:77) at org.janusgraph.graphdb.transaction.RelationConstructor$1. (RelationConstructor.java:47) at org.janusgraph.graphdb.transaction.RelationConstructor.lambda$readRelation$0(RelationConstructor.java:45) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.execute(VertexCentricQueryBuilder.java:73) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.properties(VertexCentricQueryBuilder.java:100) at org.janusgraph.graphdb.vertices.AbstractVertex.properties(AbstractVertex.java:177) at org.apache.tinkerpop.gremlin.structure.Vertex.property(Vertex.java:70) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:85) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:80) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:33:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:33:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:33:36.973Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9e7e718a-69c0-4980-9f65-d5f1237d3d59 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:33:36.973115Z InvocationID=cc67189d-7b7b-494f-817b-33e98510d17a LogTimestamp=2021-05-28T00:33:36.973156Z Entering 2021-05-28T00:33:36.973Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9e7e718a-69c0-4980-9f65-d5f1237d3d59 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:33:36.973115Z InvocationID=cc67189d-7b7b-494f-817b-33e98510d17a LogTimestamp=2021-05-28T00:33:36.973156Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:33:36.973Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:33:36.973Z InvocationID=639986ce-670e-41c4-bcd4-ed0675c825e0 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:33:36.974Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:33:36.974Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:33:36.973Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=639986ce-670e-41c4-bcd4-ed0675c825e0 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@13e2f3c5 2021-05-28T00:33:36.974Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:33:36.974Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:33:36.973Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=639986ce-670e-41c4-bcd4-ed0675c825e0 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@13e2f3c5 2021-05-28T00:33:36.974Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:33:36.974671Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:33:36.974Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:33:36.974671Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:33:36.975Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:33:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:33:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:33:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:33:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:38.668Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b75e4b-660d-473e-9fb1-fb80319f9c6d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:33:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b75e4b-660d-473e-9fb1-fb80319f9c6d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39c78e2 2021-05-28T00:33:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b75e4b-660d-473e-9fb1-fb80319f9c6d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:33:38.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b75e4b-660d-473e-9fb1-fb80319f9c6d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:33:38.676Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:33:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:42.870Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [EdgeStoreQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$1.call(BackendTransaction.java:272) at org.janusgraph.diskstorage.BackendTransaction$1.call(BackendTransaction.java:269) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.edgeStoreQuery(BackendTransaction.java:269) at org.janusgraph.graphdb.database.StandardJanusGraph.edgeQuery(StandardJanusGraph.java:436) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.lambda$null$0(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:98) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:90) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:82) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.lambda$getBasicIterator$1(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.vertices.CacheVertex.loadRelations(CacheVertex.java:67) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.getBasicIterator(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.iterator(SimpleVertexQueryProcessor.java:77) at org.janusgraph.graphdb.transaction.RelationConstructor$1. (RelationConstructor.java:47) at org.janusgraph.graphdb.transaction.RelationConstructor.lambda$readRelation$0(RelationConstructor.java:45) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.execute(VertexCentricQueryBuilder.java:73) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.properties(VertexCentricQueryBuilder.java:100) at org.janusgraph.graphdb.vertices.AbstractVertex.properties(AbstractVertex.java:177) at org.apache.tinkerpop.gremlin.structure.Vertex.property(Vertex.java:70) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:85) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:80) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 26 common frames omitted 2021-05-28T00:33:42.871Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-05-28T00:33:27.572090Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:33:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:47.072Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4ef86cb3-fc33-4161-84d7-6d6d9c6eb963 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:33:47.071922Z InvocationID=d8659920-070e-46be-af72-85236f29b04e LogTimestamp=2021-05-28T00:33:47.071966Z Entering 2021-05-28T00:33:47.072Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4ef86cb3-fc33-4161-84d7-6d6d9c6eb963 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:33:47.071922Z InvocationID=d8659920-070e-46be-af72-85236f29b04e LogTimestamp=2021-05-28T00:33:47.071966Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:33:47.072Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:33:47.072Z InvocationID=34d86a97-bc5c-40ca-8db1-82015301e732 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:33:47.073Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:33:47.073Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:33:47.072Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=34d86a97-bc5c-40ca-8db1-82015301e732 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5b85af95 2021-05-28T00:33:47.073Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:33:47.073Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:33:47.072Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=34d86a97-bc5c-40ca-8db1-82015301e732 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5b85af95 2021-05-28T00:33:47.073Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:33:47.073603Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:33:47.073Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:33:47.073603Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:33:47.074Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:33:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:33:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:33:52.117Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:33:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:33:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:02.473Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:05.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:34:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:34:08.468Z [qtp580718781-96] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "Access" USER = "(jh0003)" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." ErrorCategory=WARN RequestID=null ServiceName= TargetServiceName=null ErrorCode=900 TargetEntity= 2021-05-28T00:34:08.468Z [qtp580718781-96] INFO o.o.s.b.s.e.ComponentExceptionMapper AuditMessage=ACTION = "Access" USER = "(jh0003)" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." ErrorCategory=WARN RequestID=null ServiceName= TargetServiceName=null ErrorCode=900 TargetEntity= Error during request filter= GENERAL_ERROR 2021-05-28T00:34:08.468Z [qtp580718781-96] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "Access" USER = "(jh0003)" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." RequestID=null ErrorCategory=WARN ServiceName= TargetServiceName=null ErrorCode=900 TargetEntity= 2021-05-28T00:34:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:11.876Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:11.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e1438a8-7641-4895-b1f7-13727f6ce86b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:11.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e1438a8-7641-4895-b1f7-13727f6ce86b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dd6c186 2021-05-28T00:34:11.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e1438a8-7641-4895-b1f7-13727f6ce86b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:11.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e1438a8-7641-4895-b1f7-13727f6ce86b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:11.969Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:12.783Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:12.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3a5b028-a95a-4209-91e1-5b6dfcda496b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:12.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3a5b028-a95a-4209-91e1-5b6dfcda496b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32edcf26 2021-05-28T00:34:12.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3a5b028-a95a-4209-91e1-5b6dfcda496b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:12.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c3a5b028-a95a-4209-91e1-5b6dfcda496b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:12.868Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:13.377Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [EdgeStoreQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$1.call(BackendTransaction.java:272) at org.janusgraph.diskstorage.BackendTransaction$1.call(BackendTransaction.java:269) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.edgeStoreQuery(BackendTransaction.java:269) at org.janusgraph.graphdb.database.StandardJanusGraph.edgeQuery(StandardJanusGraph.java:436) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.lambda$null$0(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:98) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:90) at org.janusgraph.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:82) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.lambda$getBasicIterator$1(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.vertices.CacheVertex.loadRelations(CacheVertex.java:67) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.getBasicIterator(SimpleVertexQueryProcessor.java:120) at org.janusgraph.graphdb.query.vertex.SimpleVertexQueryProcessor.iterator(SimpleVertexQueryProcessor.java:77) at org.janusgraph.graphdb.transaction.RelationConstructor$1. (RelationConstructor.java:47) at org.janusgraph.graphdb.transaction.RelationConstructor.lambda$readRelation$0(RelationConstructor.java:45) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.execute(VertexCentricQueryBuilder.java:73) at org.janusgraph.graphdb.query.vertex.VertexCentricQueryBuilder.properties(VertexCentricQueryBuilder.java:100) at org.janusgraph.graphdb.vertices.AbstractVertex.properties(AbstractVertex.java:177) at org.apache.tinkerpop.gremlin.structure.Vertex.property(Vertex.java:70) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:85) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:80) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:34:13.968Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04168091-dd36-40c2-bbdb-4b24bb253a02 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04168091-dd36-40c2-bbdb-4b24bb253a02 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@191ce20c 2021-05-28T00:34:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04168091-dd36-40c2-bbdb-4b24bb253a02 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04168091-dd36-40c2-bbdb-4b24bb253a02 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:13.976Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:14.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:15.864Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:15.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c6bce3b-933d-4333-8d21-6da01aab62a2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:15.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c6bce3b-933d-4333-8d21-6da01aab62a2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1caaf005 2021-05-28T00:34:15.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c6bce3b-933d-4333-8d21-6da01aab62a2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:15.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c6bce3b-933d-4333-8d21-6da01aab62a2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:15.873Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:17.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:18.581Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a7cf345f-183e-45b4-9ff0-3edcecadfe4f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:18.580297Z InvocationID=bfb9e8bc-6f11-4d48-98ac-2a60a4262b67 LogTimestamp=2021-05-28T00:34:18.580420Z Entering 2021-05-28T00:34:18.581Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a7cf345f-183e-45b4-9ff0-3edcecadfe4f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:18.580297Z InvocationID=bfb9e8bc-6f11-4d48-98ac-2a60a4262b67 LogTimestamp=2021-05-28T00:34:18.580420Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:34:18.582Z [qtp580718781-12] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:34:18.581Z InvocationID=7702ebdc-a607-4c1c-bc88-a4728b0e82bd ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:34:18.583Z [qtp580718781-12] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=2 EndTimestamp=2021-05-28T00:34:18.583Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:18.581Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=7702ebdc-a607-4c1c-bc88-a4728b0e82bd ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@1135c18b 2021-05-28T00:34:18.583Z [qtp580718781-12] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=2 EndTimestamp=2021-05-28T00:34:18.583Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:18.581Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=7702ebdc-a607-4c1c-bc88-a4728b0e82bd ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@1135c18b 2021-05-28T00:34:18.584Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:18.583849Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:34:18.584Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:18.583849Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:34:18.584Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:34:18.869Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:18.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36c0f829-7f2d-4c00-95cc-98c0d72ec777 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:18.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36c0f829-7f2d-4c00-95cc-98c0d72ec777 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@792ec420 2021-05-28T00:34:18.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36c0f829-7f2d-4c00-95cc-98c0d72ec777 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:18.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36c0f829-7f2d-4c00-95cc-98c0d72ec777 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:18.878Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:20.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:34:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:34:21.569Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd040401-2a73-4c1c-9502-e9b41d909854 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd040401-2a73-4c1c-9502-e9b41d909854 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25e6a0dc 2021-05-28T00:34:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd040401-2a73-4c1c-9502-e9b41d909854 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:21.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd040401-2a73-4c1c-9502-e9b41d909854 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:21.576Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:26.478Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f37a76-a352-4218-a5d7-8fdc620a7019 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f37a76-a352-4218-a5d7-8fdc620a7019 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b142aee 2021-05-28T00:34:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f37a76-a352-4218-a5d7-8fdc620a7019 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:26.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f37a76-a352-4218-a5d7-8fdc620a7019 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:26.485Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:28.674Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b6d63aa5-b6ec-4def-9457-2dba730c0a02 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:28.674031Z InvocationID=742ba744-820d-4379-a39f-00896d45ccfe LogTimestamp=2021-05-28T00:34:28.674105Z Entering 2021-05-28T00:34:28.674Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b6d63aa5-b6ec-4def-9457-2dba730c0a02 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:28.674031Z InvocationID=742ba744-820d-4379-a39f-00896d45ccfe LogTimestamp=2021-05-28T00:34:28.674105Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:34:28.675Z [qtp580718781-16] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:34:28.675Z InvocationID=dc3e040e-63fd-4778-88f1-a9767495cdde ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:34:28.676Z [qtp580718781-16] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:34:28.676Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:28.675Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=dc3e040e-63fd-4778-88f1-a9767495cdde ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@410d4ada 2021-05-28T00:34:28.676Z [qtp580718781-16] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:34:28.676Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:28.675Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=dc3e040e-63fd-4778-88f1-a9767495cdde ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@410d4ada 2021-05-28T00:34:28.677Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:28.677089Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:34:28.677Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:28.677089Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:34:28.677Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:34:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:30.189Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:30.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ce6d67-da83-472c-b0c6-ee836cbdd2bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:30.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ce6d67-da83-472c-b0c6-ee836cbdd2bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10f5c351 2021-05-28T00:34:30.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ce6d67-da83-472c-b0c6-ee836cbdd2bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:30.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11ce6d67-da83-472c-b0c6-ee836cbdd2bc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:30.269Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:31.371Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:31.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0bb91c-25ac-41d5-856c-41f20209ea49 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:31.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0bb91c-25ac-41d5-856c-41f20209ea49 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ff50aab 2021-05-28T00:34:31.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0bb91c-25ac-41d5-856c-41f20209ea49 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:31.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de0bb91c-25ac-41d5-856c-41f20209ea49 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:31.379Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:32.668Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:32.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f9ba28-6602-4fe7-98b5-a1855dedaf31 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:32.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f9ba28-6602-4fe7-98b5-a1855dedaf31 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f05a9c9 2021-05-28T00:34:32.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f9ba28-6602-4fe7-98b5-a1855dedaf31 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:32.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f9ba28-6602-4fe7-98b5-a1855dedaf31 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:32.678Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:33.971Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:33.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ce53859-1b94-4fb7-91ca-5868ffc94f48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:33.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ce53859-1b94-4fb7-91ca-5868ffc94f48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@753cb19d 2021-05-28T00:34:33.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ce53859-1b94-4fb7-91ca-5868ffc94f48 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:33.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ce53859-1b94-4fb7-91ca-5868ffc94f48 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:33.978Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:34.668Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:34:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:34:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:34:35.902Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:35.902Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d7182ad-15f0-4211-beda-3ab6829389ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:35.902Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d7182ad-15f0-4211-beda-3ab6829389ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b6d1b20 2021-05-28T00:34:35.902Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d7182ad-15f0-4211-beda-3ab6829389ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:35.902Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d7182ad-15f0-4211-beda-3ab6829389ca PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:35.908Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:34:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:34:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:34:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:38.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:34:40.276Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ea49fde6-c759-4021-bc3c-67af4c08ead5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:40.276216Z InvocationID=46b02a1a-dfd7-4c49-81a7-0c9507432f27 LogTimestamp=2021-05-28T00:34:40.276305Z Entering 2021-05-28T00:34:40.277Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ea49fde6-c759-4021-bc3c-67af4c08ead5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:40.276216Z InvocationID=46b02a1a-dfd7-4c49-81a7-0c9507432f27 LogTimestamp=2021-05-28T00:34:40.276305Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:34:40.277Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:34:40.277Z InvocationID=f5bcb6a8-7c22-48e7-af33-073ee645e31d ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:34:40.278Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:34:40.278Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:40.277Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=f5bcb6a8-7c22-48e7-af33-073ee645e31d ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@3ce200fe 2021-05-28T00:34:40.278Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:34:40.278Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:40.277Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=f5bcb6a8-7c22-48e7-af33-073ee645e31d ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@3ce200fe 2021-05-28T00:34:40.279Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:40.279053Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:34:40.279Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:40.279053Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:34:40.279Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:34:40.503Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:40.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5e44a6a-9233-4533-8633-680167f80377 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:40.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5e44a6a-9233-4533-8633-680167f80377 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c7af0b4 2021-05-28T00:34:40.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5e44a6a-9233-4533-8633-680167f80377 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:40.503Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5e44a6a-9233-4533-8633-680167f80377 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:40.506Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e5e44a6a-9233-4533-8633-680167f80377 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:40.507Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e5e44a6a-9233-4533-8633-680167f80377 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:40.575Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:40.756Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:34:41.472Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:41.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7091ccd1-9c38-468e-b1b5-f109e70ea3ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:41.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7091ccd1-9c38-468e-b1b5-f109e70ea3ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e135513 2021-05-28T00:34:41.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7091ccd1-9c38-468e-b1b5-f109e70ea3ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:41.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7091ccd1-9c38-468e-b1b5-f109e70ea3ca PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:41.475Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7091ccd1-9c38-468e-b1b5-f109e70ea3ca ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:41.475Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=7091ccd1-9c38-468e-b1b5-f109e70ea3ca ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:34:41.479Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:43.671Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4ae09-b1df-4316-b73b-26cecaab153d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4ae09-b1df-4316-b73b-26cecaab153d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e33b715 2021-05-28T00:34:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4ae09-b1df-4316-b73b-26cecaab153d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4ae09-b1df-4316-b73b-26cecaab153d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:43.674Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f5f4ae09-b1df-4316-b73b-26cecaab153d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:43.674Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f5f4ae09-b1df-4316-b73b-26cecaab153d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:43.678Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:44.768Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:44.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dce920-4a44-4e6e-9960-7f7a174d3914 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:44.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dce920-4a44-4e6e-9960-7f7a174d3914 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d2401e8 2021-05-28T00:34:44.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dce920-4a44-4e6e-9960-7f7a174d3914 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:44.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dce920-4a44-4e6e-9960-7f7a174d3914 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:44.776Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:49.169Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:49.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ecca14a-eea7-47e6-a0ee-b5396ad1a749 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:49.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ecca14a-eea7-47e6-a0ee-b5396ad1a749 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4670a3a8 2021-05-28T00:34:49.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ecca14a-eea7-47e6-a0ee-b5396ad1a749 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:49.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ecca14a-eea7-47e6-a0ee-b5396ad1a749 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:49.177Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:34:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:34:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:34:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:34:50.874Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=877e101b-ff32-42d1-ad60-e0cbc3fa9081 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:50.874392Z InvocationID=db199e99-e721-445f-8f57-cdefecb27534 LogTimestamp=2021-05-28T00:34:50.874492Z Entering 2021-05-28T00:34:50.874Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=877e101b-ff32-42d1-ad60-e0cbc3fa9081 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.178 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.178 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:34:50.874392Z InvocationID=db199e99-e721-445f-8f57-cdefecb27534 LogTimestamp=2021-05-28T00:34:50.874492Z localAddr=10.242.102.221 remoteAddr=10.242.170.178 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:34:50.876Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 TargetServiceName=null EntryTimestamp=2021-05-28T00:34:50.875Z InvocationID=e7a188f2-0267-4d6e-b5b5-42cff77acff4 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:34:50.876Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:34:50.876Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:50.875Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=e7a188f2-0267-4d6e-b5b5-42cff77acff4 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@3f7e0832 2021-05-28T00:34:50.876Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:34:50.876Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.178 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:34:50.875Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=e7a188f2-0267-4d6e-b5b5-42cff77acff4 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@3f7e0832 2021-05-28T00:34:50.876Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:50.876794Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:34:50.876Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:34:50.876794Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:34:50.877Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="500" 2021-05-28T00:34:51.568Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:51.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=072c38d0-6611-4173-9b23-09630978d503 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:51.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=072c38d0-6611-4173-9b23-09630978d503 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20fcdb8 2021-05-28T00:34:51.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=072c38d0-6611-4173-9b23-09630978d503 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:51.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=072c38d0-6611-4173-9b23-09630978d503 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:51.575Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:52.118Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:34:57.669Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:57.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211e640c-8bb8-4df0-a364-ea221e1b2d6a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:57.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211e640c-8bb8-4df0-a364-ea221e1b2d6a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51d759f3 2021-05-28T00:34:57.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211e640c-8bb8-4df0-a364-ea221e1b2d6a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:57.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211e640c-8bb8-4df0-a364-ea221e1b2d6a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:57.671Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:34:57.675Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=211e640c-8bb8-4df0-a364-ea221e1b2d6a ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:57.675Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=211e640c-8bb8-4df0-a364-ea221e1b2d6a ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:34:57.678Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:58.372Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:58.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205b1ecb-4e0d-4855-8f58-836abcd53ded PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:58.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205b1ecb-4e0d-4855-8f58-836abcd53ded PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cb13960 2021-05-28T00:34:58.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205b1ecb-4e0d-4855-8f58-836abcd53ded PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:58.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205b1ecb-4e0d-4855-8f58-836abcd53ded PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:58.378Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=205b1ecb-4e0d-4855-8f58-836abcd53ded ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:58.378Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=205b1ecb-4e0d-4855-8f58-836abcd53ded ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:34:58.388Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:59.163Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:34:59.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4db2f9fa-df01-4085-887e-4c0278c696b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:34:59.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4db2f9fa-df01-4085-887e-4c0278c696b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f9fb4bd 2021-05-28T00:34:59.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4db2f9fa-df01-4085-887e-4c0278c696b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:34:59.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4db2f9fa-df01-4085-887e-4c0278c696b7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:34:59.176Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:02.073Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4b5715-7120-4f09-931c-e144e0e2fdac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4b5715-7120-4f09-931c-e144e0e2fdac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2381d0fb 2021-05-28T00:35:02.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4b5715-7120-4f09-931c-e144e0e2fdac PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:02.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4b5715-7120-4f09-931c-e144e0e2fdac PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:02.084Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:03.269Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:03.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ced0833-c4f4-44fd-a5fb-8995d0e77c8c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:03.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ced0833-c4f4-44fd-a5fb-8995d0e77c8c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48720db9 2021-05-28T00:35:03.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ced0833-c4f4-44fd-a5fb-8995d0e77c8c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:03.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ced0833-c4f4-44fd-a5fb-8995d0e77c8c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:03.367Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:05.499Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:05.499Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1a6ed42-b680-45b8-a836-a9abfd8d661c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:05.500Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1a6ed42-b680-45b8-a836-a9abfd8d661c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d9d57ee 2021-05-28T00:35:05.500Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1a6ed42-b680-45b8-a836-a9abfd8d661c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:05.500Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1a6ed42-b680-45b8-a836-a9abfd8d661c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:35:05.508Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:35:06.709Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:06.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a472d079-73f9-4f3b-a60f-26b6ed083313 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:06.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a472d079-73f9-4f3b-a60f-26b6ed083313 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f053abc 2021-05-28T00:35:06.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a472d079-73f9-4f3b-a60f-26b6ed083313 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:06.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a472d079-73f9-4f3b-a60f-26b6ed083313 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:06.716Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:35:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:35:09.685Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE 2021-05-28T00:35:09.686Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 256000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:35:13.671Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:35:14.068Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:14.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a60821c5-84d1-4e14-b23c-7640a465f134 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:14.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a60821c5-84d1-4e14-b23c-7640a465f134 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@522e2baf 2021-05-28T00:35:14.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a60821c5-84d1-4e14-b23c-7640a465f134 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:14.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a60821c5-84d1-4e14-b23c-7640a465f134 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:14.076Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:14.864Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:14.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65c2ae2f-337e-4823-82d4-91ddcede2ca9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:14.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65c2ae2f-337e-4823-82d4-91ddcede2ca9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a69eb96 2021-05-28T00:35:14.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65c2ae2f-337e-4823-82d4-91ddcede2ca9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:14.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65c2ae2f-337e-4823-82d4-91ddcede2ca9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:14.871Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:17.769Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:17.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5b3d2b-868b-4183-becf-9b932c1c1e89 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:17.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5b3d2b-868b-4183-becf-9b932c1c1e89 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dfcff12 2021-05-28T00:35:17.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5b3d2b-868b-4183-becf-9b932c1c1e89 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:17.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5b3d2b-868b-4183-becf-9b932c1c1e89 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:17.777Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:35:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:35:21.189Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:21.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe2d83c-97fa-49e2-8203-ef0037493b6d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:21.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe2d83c-97fa-49e2-8203-ef0037493b6d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bfaf2cb 2021-05-28T00:35:21.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe2d83c-97fa-49e2-8203-ef0037493b6d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:21.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe2d83c-97fa-49e2-8203-ef0037493b6d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:21.268Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:23.168Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:23.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fda6c73-d8b5-4f62-8c45-811b4ac07d92 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:23.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fda6c73-d8b5-4f62-8c45-811b4ac07d92 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@482acf48 2021-05-28T00:35:23.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fda6c73-d8b5-4f62-8c45-811b4ac07d92 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:23.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fda6c73-d8b5-4f62-8c45-811b4ac07d92 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:23.176Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:26.990Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:26.990Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5553d4f9-460f-4ead-bfef-72b54314e29f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:26.990Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5553d4f9-460f-4ead-bfef-72b54314e29f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ddaa880 2021-05-28T00:35:26.990Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5553d4f9-460f-4ead-bfef-72b54314e29f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:26.990Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5553d4f9-460f-4ead-bfef-72b54314e29f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:26.998Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:28.268Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:28.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d9257a6-3b2a-4cf3-892c-318e56c11355 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:28.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d9257a6-3b2a-4cf3-892c-318e56c11355 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5704d246 2021-05-28T00:35:28.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d9257a6-3b2a-4cf3-892c-318e56c11355 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:28.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d9257a6-3b2a-4cf3-892c-318e56c11355 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:28.275Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:35:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:35:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:35:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:35:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:35:41.970Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:41.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae5ba6b6-d4c2-4cc2-b837-a24b5ca5fa0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:41.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae5ba6b6-d4c2-4cc2-b837-a24b5ca5fa0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b541ec7 2021-05-28T00:35:41.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae5ba6b6-d4c2-4cc2-b837-a24b5ca5fa0d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:41.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae5ba6b6-d4c2-4cc2-b837-a24b5ca5fa0d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:41.982Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:44.176Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:44.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82797e68-98e0-4b52-8a7b-210461d3cf07 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:44.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82797e68-98e0-4b52-8a7b-210461d3cf07 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@751acc6a 2021-05-28T00:35:44.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82797e68-98e0-4b52-8a7b-210461d3cf07 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:44.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82797e68-98e0-4b52-8a7b-210461d3cf07 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:44.269Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:47.877Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:47.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0702d8fd-6e4b-4596-9af7-6c17508b30e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:47.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0702d8fd-6e4b-4596-9af7-6c17508b30e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ddcf4e1 2021-05-28T00:35:47.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0702d8fd-6e4b-4596-9af7-6c17508b30e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:47.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0702d8fd-6e4b-4596-9af7-6c17508b30e3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:47.885Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:35:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:35:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:35:50.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e0b90b5-dba9-45a4-aa8b-e4b23d163eae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:50.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e0b90b5-dba9-45a4-aa8b-e4b23d163eae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c3b85fe 2021-05-28T00:35:50.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e0b90b5-dba9-45a4-aa8b-e4b23d163eae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:50.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e0b90b5-dba9-45a4-aa8b-e4b23d163eae PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:52.123Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:35:52.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6474831-38d1-437e-9a9d-0f65ede2681a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:52.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6474831-38d1-437e-9a9d-0f65ede2681a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17520a98 2021-05-28T00:35:52.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6474831-38d1-437e-9a9d-0f65ede2681a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:52.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6474831-38d1-437e-9a9d-0f65ede2681a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85c9c6ef-5c27-46ee-aa47-7368eb58a3d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85c9c6ef-5c27-46ee-aa47-7368eb58a3d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3488fd84 2021-05-28T00:35:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85c9c6ef-5c27-46ee-aa47-7368eb58a3d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:53.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85c9c6ef-5c27-46ee-aa47-7368eb58a3d8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:54.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9d1d6a3-b4e9-4248-aa42-574811332798 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:54.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9d1d6a3-b4e9-4248-aa42-574811332798 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e397c08 2021-05-28T00:35:54.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9d1d6a3-b4e9-4248-aa42-574811332798 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:54.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9d1d6a3-b4e9-4248-aa42-574811332798 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:54.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f041bf6-c2e6-49b6-beb6-d8e72b153c84 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:54.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f041bf6-c2e6-49b6-beb6-d8e72b153c84 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ead0244 2021-05-28T00:35:54.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f041bf6-c2e6-49b6-beb6-d8e72b153c84 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:54.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f041bf6-c2e6-49b6-beb6-d8e72b153c84 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:55.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca0ed5d-4bae-43b6-aa39-cf1afb0d19da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:55.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca0ed5d-4bae-43b6-aa39-cf1afb0d19da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79acc1c3 2021-05-28T00:35:55.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca0ed5d-4bae-43b6-aa39-cf1afb0d19da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:55.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca0ed5d-4bae-43b6-aa39-cf1afb0d19da PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:56.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5159a891-eebf-4e4e-be71-184f19db02d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:56.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5159a891-eebf-4e4e-be71-184f19db02d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@abb77af 2021-05-28T00:35:56.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5159a891-eebf-4e4e-be71-184f19db02d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:56.529Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5159a891-eebf-4e4e-be71-184f19db02d5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:57.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ce50a70-cef4-4515-90db-a1014a73ea38 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:57.566Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ce50a70-cef4-4515-90db-a1014a73ea38 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f5793cc 2021-05-28T00:35:57.566Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ce50a70-cef4-4515-90db-a1014a73ea38 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:57.566Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ce50a70-cef4-4515-90db-a1014a73ea38 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:58.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59cf070f-88e6-49ba-8b7b-4af78d7cc176 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:58.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59cf070f-88e6-49ba-8b7b-4af78d7cc176 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29fe0940 2021-05-28T00:35:58.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59cf070f-88e6-49ba-8b7b-4af78d7cc176 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:58.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59cf070f-88e6-49ba-8b7b-4af78d7cc176 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:59.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d86cdb5-62ba-4640-a483-1a7c1a739fc0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:59.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d86cdb5-62ba-4640-a483-1a7c1a739fc0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36437765 2021-05-28T00:35:59.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d86cdb5-62ba-4640-a483-1a7c1a739fc0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:59.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d86cdb5-62ba-4640-a483-1a7c1a739fc0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:35:59.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae52e7a-4794-49a8-ae85-17c00fe78758 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:35:59.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae52e7a-4794-49a8-ae85-17c00fe78758 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6794780d 2021-05-28T00:35:59.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae52e7a-4794-49a8-ae85-17c00fe78758 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:35:59.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae52e7a-4794-49a8-ae85-17c00fe78758 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:00.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89c7d1bd-dec5-4e11-88ed-ada5602e6e4e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:00.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89c7d1bd-dec5-4e11-88ed-ada5602e6e4e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64de5ec0 2021-05-28T00:36:00.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89c7d1bd-dec5-4e11-88ed-ada5602e6e4e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:00.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89c7d1bd-dec5-4e11-88ed-ada5602e6e4e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:01.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18113e72-4131-4dca-beb6-9a63ef1db3c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:01.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18113e72-4131-4dca-beb6-9a63ef1db3c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3db3ef12 2021-05-28T00:36:01.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18113e72-4131-4dca-beb6-9a63ef1db3c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:01.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18113e72-4131-4dca-beb6-9a63ef1db3c7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:03.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ac40a80-705f-4c8f-b072-cc9fb68651bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:03.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ac40a80-705f-4c8f-b072-cc9fb68651bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10665f9e 2021-05-28T00:36:03.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ac40a80-705f-4c8f-b072-cc9fb68651bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:03.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ac40a80-705f-4c8f-b072-cc9fb68651bc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:04.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0a7ea7-01a7-4213-89c2-928ecbdff2d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:04.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0a7ea7-01a7-4213-89c2-928ecbdff2d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5476098f 2021-05-28T00:36:04.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0a7ea7-01a7-4213-89c2-928ecbdff2d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:04.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c0a7ea7-01a7-4213-89c2-928ecbdff2d4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:36:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:36:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be8a2d17-d4a6-433e-b1db-2f0658ee08f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be8a2d17-d4a6-433e-b1db-2f0658ee08f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42c8659f 2021-05-28T00:36:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be8a2d17-d4a6-433e-b1db-2f0658ee08f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:07.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be8a2d17-d4a6-433e-b1db-2f0658ee08f1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:10.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8708efb-13b1-4a80-a843-1faac395305e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:10.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8708efb-13b1-4a80-a843-1faac395305e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6021d9ab 2021-05-28T00:36:10.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8708efb-13b1-4a80-a843-1faac395305e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:10.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8708efb-13b1-4a80-a843-1faac395305e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:14.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67fb4115-d3bb-45c2-8807-4add36f6be99 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:14.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67fb4115-d3bb-45c2-8807-4add36f6be99 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4219e09e 2021-05-28T00:36:14.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67fb4115-d3bb-45c2-8807-4add36f6be99 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:14.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67fb4115-d3bb-45c2-8807-4add36f6be99 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:16.287Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c141f50-74c8-4258-9c5a-d961242c5a67 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:16.287Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c141f50-74c8-4258-9c5a-d961242c5a67 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2647b323 2021-05-28T00:36:16.287Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c141f50-74c8-4258-9c5a-d961242c5a67 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:16.287Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c141f50-74c8-4258-9c5a-d961242c5a67 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:19.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f15f8231-4d2b-4851-8efd-3a4db917c9c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:19.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f15f8231-4d2b-4851-8efd-3a4db917c9c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27a2dee9 2021-05-28T00:36:19.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f15f8231-4d2b-4851-8efd-3a4db917c9c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:19.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f15f8231-4d2b-4851-8efd-3a4db917c9c3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:36:20.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:36:23.183Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e56d0239-8343-449c-8eb7-5fcdbbd534dc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:23.183Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e56d0239-8343-449c-8eb7-5fcdbbd534dc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4920a737 2021-05-28T00:36:23.183Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e56d0239-8343-449c-8eb7-5fcdbbd534dc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:23.183Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e56d0239-8343-449c-8eb7-5fcdbbd534dc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:26.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cf091e6-37fd-4958-894c-356c0ea36984 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:26.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cf091e6-37fd-4958-894c-356c0ea36984 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4430b39d 2021-05-28T00:36:26.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cf091e6-37fd-4958-894c-356c0ea36984 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:26.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cf091e6-37fd-4958-894c-356c0ea36984 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:28.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e49294e-91de-4fd0-9c2d-e210fbf4b7ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:28.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e49294e-91de-4fd0-9c2d-e210fbf4b7ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f75de93 2021-05-28T00:36:28.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e49294e-91de-4fd0-9c2d-e210fbf4b7ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:28.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e49294e-91de-4fd0-9c2d-e210fbf4b7ed PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:31.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d286b40-2230-4c34-bd1f-4dc2592e8c8a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:31.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d286b40-2230-4c34-bd1f-4dc2592e8c8a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e911602 2021-05-28T00:36:31.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d286b40-2230-4c34-bd1f-4dc2592e8c8a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:31.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d286b40-2230-4c34-bd1f-4dc2592e8c8a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:35.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31f20a56-21cd-4bbc-825b-03397a8b4457 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:35.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31f20a56-21cd-4bbc-825b-03397a8b4457 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22440df8 2021-05-28T00:36:35.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31f20a56-21cd-4bbc-825b-03397a8b4457 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:35.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31f20a56-21cd-4bbc-825b-03397a8b4457 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:35.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:36:35.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:36:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:36:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:36:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:36:37.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08fa3c96-ad92-4c36-acc0-48ccecf84e78 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:37.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08fa3c96-ad92-4c36-acc0-48ccecf84e78 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63a9fe76 2021-05-28T00:36:37.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08fa3c96-ad92-4c36-acc0-48ccecf84e78 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:37.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08fa3c96-ad92-4c36-acc0-48ccecf84e78 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:40.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04d50584-5b33-47ac-aa05-98c3a9c1066f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:40.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04d50584-5b33-47ac-aa05-98c3a9c1066f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12d935d6 2021-05-28T00:36:40.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04d50584-5b33-47ac-aa05-98c3a9c1066f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:40.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04d50584-5b33-47ac-aa05-98c3a9c1066f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf05fdf8-3e39-445b-9c11-75666c5299f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf05fdf8-3e39-445b-9c11-75666c5299f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@749e0089 2021-05-28T00:36:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf05fdf8-3e39-445b-9c11-75666c5299f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf05fdf8-3e39-445b-9c11-75666c5299f9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:46.193Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47f5a3b9-7026-4bb5-ade4-8681e638163a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:46.193Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47f5a3b9-7026-4bb5-ade4-8681e638163a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76b7f82f 2021-05-28T00:36:46.193Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47f5a3b9-7026-4bb5-ade4-8681e638163a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:46.193Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47f5a3b9-7026-4bb5-ade4-8681e638163a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:49.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f1e0184-84b4-4339-831e-220472cc355e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:49.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f1e0184-84b4-4339-831e-220472cc355e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ac1f2c1 2021-05-28T00:36:49.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f1e0184-84b4-4339-831e-220472cc355e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:49.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f1e0184-84b4-4339-831e-220472cc355e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:36:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:36:51.833Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:36:52.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d7d2c1-b6da-48f9-b934-87865f39cc3c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:52.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d7d2c1-b6da-48f9-b934-87865f39cc3c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ed1bca 2021-05-28T00:36:52.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d7d2c1-b6da-48f9-b934-87865f39cc3c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:52.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d7d2c1-b6da-48f9-b934-87865f39cc3c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:55.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=959c8fb8-292d-499f-889b-6e11a0ad26a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:55.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=959c8fb8-292d-499f-889b-6e11a0ad26a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@573d936 2021-05-28T00:36:55.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=959c8fb8-292d-499f-889b-6e11a0ad26a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:55.493Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=959c8fb8-292d-499f-889b-6e11a0ad26a8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:36:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4515e242-f718-482d-9319-9791240f8022 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:36:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4515e242-f718-482d-9319-9791240f8022 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c4c3f37 2021-05-28T00:36:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4515e242-f718-482d-9319-9791240f8022 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:36:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4515e242-f718-482d-9319-9791240f8022 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:02.406Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94366cc0-c722-499d-a836-04dfe0a6895c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:02.406Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94366cc0-c722-499d-a836-04dfe0a6895c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19ce8b85 2021-05-28T00:37:02.407Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94366cc0-c722-499d-a836-04dfe0a6895c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:02.407Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94366cc0-c722-499d-a836-04dfe0a6895c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7986579f-7b3f-4235-b46e-573cf9286a5b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7986579f-7b3f-4235-b46e-573cf9286a5b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7014f03d 2021-05-28T00:37:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7986579f-7b3f-4235-b46e-573cf9286a5b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7986579f-7b3f-4235-b46e-573cf9286a5b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:05.509Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:37:05.524Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:37:07.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb5521f-e62d-410b-a6df-6c75d8505b22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:07.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb5521f-e62d-410b-a6df-6c75d8505b22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bb53d60 2021-05-28T00:37:07.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb5521f-e62d-410b-a6df-6c75d8505b22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:07.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb5521f-e62d-410b-a6df-6c75d8505b22 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:10.918Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8db5f4c2-6126-48be-9f44-30200037de37 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:10.918Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8db5f4c2-6126-48be-9f44-30200037de37 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46ad12a6 2021-05-28T00:37:10.918Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8db5f4c2-6126-48be-9f44-30200037de37 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:10.918Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8db5f4c2-6126-48be-9f44-30200037de37 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:16.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b50ef09-3b1d-4f22-94c2-0a546f68af5d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:16.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b50ef09-3b1d-4f22-94c2-0a546f68af5d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b522b55 2021-05-28T00:37:16.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b50ef09-3b1d-4f22-94c2-0a546f68af5d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:16.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b50ef09-3b1d-4f22-94c2-0a546f68af5d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:16.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b0568f-77cc-4db5-a495-35c702463f26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:16.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b0568f-77cc-4db5-a495-35c702463f26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c90a31a 2021-05-28T00:37:16.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b0568f-77cc-4db5-a495-35c702463f26 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:16.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2b0568f-77cc-4db5-a495-35c702463f26 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:37:20.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:37:20.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2da21b2a-0f27-4eb3-bd45-9b217c22450f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:20.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2da21b2a-0f27-4eb3-bd45-9b217c22450f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e8ccfd5 2021-05-28T00:37:20.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2da21b2a-0f27-4eb3-bd45-9b217c22450f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:20.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2da21b2a-0f27-4eb3-bd45-9b217c22450f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:22.797Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb73356d-a687-4911-b748-97e547771617 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:22.797Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb73356d-a687-4911-b748-97e547771617 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@109ad280 2021-05-28T00:37:22.797Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb73356d-a687-4911-b748-97e547771617 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:22.797Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb73356d-a687-4911-b748-97e547771617 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:25.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc150ea5-977a-4a8d-83d7-dc0ace892f5e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:25.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc150ea5-977a-4a8d-83d7-dc0ace892f5e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a285b45 2021-05-28T00:37:25.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc150ea5-977a-4a8d-83d7-dc0ace892f5e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:25.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc150ea5-977a-4a8d-83d7-dc0ace892f5e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f7f47cc-e343-4dc7-ac67-e0135c3a4556 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f7f47cc-e343-4dc7-ac67-e0135c3a4556 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5359b569 2021-05-28T00:37:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f7f47cc-e343-4dc7-ac67-e0135c3a4556 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f7f47cc-e343-4dc7-ac67-e0135c3a4556 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:32.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db2857a-0129-4aaa-a353-a5b3406569c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:32.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db2857a-0129-4aaa-a353-a5b3406569c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bf32ab0 2021-05-28T00:37:32.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db2857a-0129-4aaa-a353-a5b3406569c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:32.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db2857a-0129-4aaa-a353-a5b3406569c4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:37:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:37:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:37:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:37:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:37:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:37:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:37:37.575Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:37:38.093Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c98f6c-2b53-4a98-b670-1b8a4737cb1f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:38.093Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c98f6c-2b53-4a98-b670-1b8a4737cb1f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d3bf79c 2021-05-28T00:37:38.093Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c98f6c-2b53-4a98-b670-1b8a4737cb1f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:38.093Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c98f6c-2b53-4a98-b670-1b8a4737cb1f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:38.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee6efcd1-c5d1-4416-a807-cf423272fdb5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:38.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee6efcd1-c5d1-4416-a807-cf423272fdb5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25216f0d 2021-05-28T00:37:38.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee6efcd1-c5d1-4416-a807-cf423272fdb5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:38.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee6efcd1-c5d1-4416-a807-cf423272fdb5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c38b57-e7ae-4cb2-8044-a1695694262b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c38b57-e7ae-4cb2-8044-a1695694262b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b4d643d 2021-05-28T00:37:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c38b57-e7ae-4cb2-8044-a1695694262b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c38b57-e7ae-4cb2-8044-a1695694262b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:44.483Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c60be4c0-0f8c-41db-a96d-c6178863cfcf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:44.483Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c60be4c0-0f8c-41db-a96d-c6178863cfcf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29797f4e 2021-05-28T00:37:44.483Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c60be4c0-0f8c-41db-a96d-c6178863cfcf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:44.483Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c60be4c0-0f8c-41db-a96d-c6178863cfcf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:47.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51473270-1147-416b-9388-005bb6c28c35 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:47.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51473270-1147-416b-9388-005bb6c28c35 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3beceaea 2021-05-28T00:37:47.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51473270-1147-416b-9388-005bb6c28c35 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:47.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51473270-1147-416b-9388-005bb6c28c35 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:49.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d61aae33-2f84-459c-9994-8d616a97c2ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:49.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d61aae33-2f84-459c-9994-8d616a97c2ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@159bb39d 2021-05-28T00:37:49.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d61aae33-2f84-459c-9994-8d616a97c2ce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:49.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d61aae33-2f84-459c-9994-8d616a97c2ce PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:37:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:37:52.120Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:37:53.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6629d47-85b6-4adc-abef-fad3884cfdeb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:53.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6629d47-85b6-4adc-abef-fad3884cfdeb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4527519f 2021-05-28T00:37:53.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6629d47-85b6-4adc-abef-fad3884cfdeb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:53.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6629d47-85b6-4adc-abef-fad3884cfdeb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:56.101Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965dac6d-a0b8-433f-ad20-f372a8682153 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:56.101Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965dac6d-a0b8-433f-ad20-f372a8682153 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2746c57a 2021-05-28T00:37:56.101Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965dac6d-a0b8-433f-ad20-f372a8682153 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:56.102Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=965dac6d-a0b8-433f-ad20-f372a8682153 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:37:58.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ade78f1-d972-426a-9a7b-186fde318210 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:37:58.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ade78f1-d972-426a-9a7b-186fde318210 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ba1c0c2 2021-05-28T00:37:58.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ade78f1-d972-426a-9a7b-186fde318210 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:37:58.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ade78f1-d972-426a-9a7b-186fde318210 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:38:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:38:04.942Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:38:04.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e5d558-1bc0-4732-86d0-6652495c814f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:04.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e5d558-1bc0-4732-86d0-6652495c814f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@336e9370 2021-05-28T00:38:04.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e5d558-1bc0-4732-86d0-6652495c814f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:04.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77e5d558-1bc0-4732-86d0-6652495c814f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:38:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:38:07.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3919a7e5-1458-460e-b506-265bd30b7b9c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:07.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3919a7e5-1458-460e-b506-265bd30b7b9c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c3a872b 2021-05-28T00:38:07.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3919a7e5-1458-460e-b506-265bd30b7b9c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:07.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3919a7e5-1458-460e-b506-265bd30b7b9c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:08.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51c3742-769f-4b5b-806b-e7ec4e5cc507 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:08.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51c3742-769f-4b5b-806b-e7ec4e5cc507 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d46f6c3 2021-05-28T00:38:08.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51c3742-769f-4b5b-806b-e7ec4e5cc507 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:08.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51c3742-769f-4b5b-806b-e7ec4e5cc507 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:11.402Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=913e57bf-0bf2-4b99-bef1-8e7cac011f41 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:11.402Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=913e57bf-0bf2-4b99-bef1-8e7cac011f41 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28948a79 2021-05-28T00:38:11.402Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=913e57bf-0bf2-4b99-bef1-8e7cac011f41 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:11.402Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=913e57bf-0bf2-4b99-bef1-8e7cac011f41 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec1584b6-f8bd-47c9-9e16-ad1f10da4b2c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec1584b6-f8bd-47c9-9e16-ad1f10da4b2c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fa66908 2021-05-28T00:38:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec1584b6-f8bd-47c9-9e16-ad1f10da4b2c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec1584b6-f8bd-47c9-9e16-ad1f10da4b2c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:17.567Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0c9119db-f728-45a5-8c5b-ee12f71293d5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:17.566894Z InvocationID=fdfa35ba-c634-43b4-a48a-1f676a04779c LogTimestamp=2021-05-28T00:38:17.567036Z Entering 2021-05-28T00:38:17.568Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0c9119db-f728-45a5-8c5b-ee12f71293d5 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:17.566894Z InvocationID=fdfa35ba-c634-43b4-a48a-1f676a04779c LogTimestamp=2021-05-28T00:38:17.567036Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:38:17.666Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=97 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:38:17.568Z InvocationID=da4911d5-ce28-4bf2-8bf0-25fc4049876c LogTimestamp=2021-05-28T00:38:17.665953Z ResponseDesc=OK Exiting. 2021-05-28T00:38:17.667Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:38:17.774Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:17.774521Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:17.774Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:17.774521Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:17.775Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:38:17.864Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "Access" USER = "(jh0003)" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." RequestID=bba56418-8309-41b7-9c90-b2ae981d0ace InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS TargetEntity= RemoteHost=10.242.170.179 ErrorCategory=WARN ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:38:17.863949Z InvocationID=76651996-bccf-47cd-aa7d-f0078014ed4b ErrorCode=900 LogTimestamp=2021-05-28T00:38:17.863998Z Entering 2021-05-28T00:38:17.864Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter AuditMessage=ACTION = "Access" USER = "(jh0003)" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." RequestID=bba56418-8309-41b7-9c90-b2ae981d0ace InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 TargetEntity= RemoteHost=10.242.170.179 ErrorCategory=WARN ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:38:17.863949Z InvocationID=76651996-bccf-47cd-aa7d-f0078014ed4b ErrorCode=900 LogTimestamp=2021-05-28T00:38:17.863998Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:38:17.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc89dd8-d6b5-45d1-9517-75fd1546f625 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:17.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc89dd8-d6b5-45d1-9517-75fd1546f625 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@773da808 2021-05-28T00:38:17.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc89dd8-d6b5-45d1-9517-75fd1546f625 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:17.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc89dd8-d6b5-45d1-9517-75fd1546f625 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:18.068Z [qtp580718781-96] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:38:17.864Z InvocationID=097583cc-146c-4723-83be-c811f2a15af8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@15e58f01', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 202ms 2021-05-28T00:38:18.070Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=205 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:38:17.864Z InvocationID=097583cc-146c-4723-83be-c811f2a15af8 ErrorCode=409 LogTimestamp=2021-05-28T00:38:18.069601Z ResponseDesc=Conflict Exiting. 2021-05-28T00:38:18.070Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:38:18.264Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=191524 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:35:06.740Z InvocationID=8dee67ea-73ae-40f4-9814-da6845a4d8b8 ErrorCode=500 LogTimestamp=2021-05-28T00:38:18.264622Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:18.265Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:38:18.366Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f16f31b5-f803-4661-be1e-d6c10e2cad66 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:18.366737Z InvocationID=8c1f0471-c301-4bd9-a518-32bfbbbe43b1 LogTimestamp=2021-05-28T00:38:18.366802Z Entering 2021-05-28T00:38:18.367Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f16f31b5-f803-4661-be1e-d6c10e2cad66 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:18.366737Z InvocationID=8c1f0471-c301-4bd9-a518-32bfbbbe43b1 LogTimestamp=2021-05-28T00:38:18.366802Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:38:18.477Z [qtp580718781-11] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:38:18.367Z InvocationID=1e032ef2-c2ee-4cda-8f69-32fda30a67e0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@15e58f01', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 109ms 2021-05-28T00:38:18.478Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=110 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:38:18.367Z InvocationID=1e032ef2-c2ee-4cda-8f69-32fda30a67e0 ErrorCode=400 LogTimestamp=2021-05-28T00:38:18.477989Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:38:18.478Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:38:18.570Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:18.570541Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:18.570Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:18.570541Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:18.571Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:38:18.763Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c5ae2579-0d31-4c54-a1fe-b3263b7550a9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:18.763004Z InvocationID=4b312741-7b7e-45a4-abca-200284326ff4 LogTimestamp=2021-05-28T00:38:18.763059Z Entering 2021-05-28T00:38:18.763Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c5ae2579-0d31-4c54-a1fe-b3263b7550a9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:18.763004Z InvocationID=4b312741-7b7e-45a4-abca-200284326ff4 LogTimestamp=2021-05-28T00:38:18.763059Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:38:19.461Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:38:18.763Z InvocationID=0ce57a84-944a-4979-a5d1-6029fd805b02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@15e58f01', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 696ms 2021-05-28T00:38:19.462Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=698 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:38:18.763Z InvocationID=0ce57a84-944a-4979-a5d1-6029fd805b02 ErrorCode=400 LogTimestamp=2021-05-28T00:38:19.461869Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:38:19.462Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:38:19.475Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=184650 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:35:14.824Z InvocationID=a4c9e367-ad92-4a66-91a1-d89e31a6b761 ErrorCode=500 LogTimestamp=2021-05-28T00:38:19.474659Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:19.475Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:38:19.674Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b9d0e04d-7f04-43a9-a80b-aaebd24ddac8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:19.674581Z InvocationID=74f80ae4-84c8-45d7-87a3-b8b81129a496 LogTimestamp=2021-05-28T00:38:19.674642Z Entering 2021-05-28T00:38:19.674Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b9d0e04d-7f04-43a9-a80b-aaebd24ddac8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:19.674581Z InvocationID=74f80ae4-84c8-45d7-87a3-b8b81129a496 LogTimestamp=2021-05-28T00:38:19.674642Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:38:19.776Z [qtp580718781-17] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:38:19.675Z InvocationID=51745925-cb5f-4076-ba00-5825b041c0a5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@15e58f01', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 100ms 2021-05-28T00:38:19.777Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=102 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:38:19.675Z InvocationID=51745925-cb5f-4076-ba00-5825b041c0a5 ErrorCode=409 LogTimestamp=2021-05-28T00:38:19.777205Z ResponseDesc=Conflict Exiting. 2021-05-28T00:38:19.777Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:38:19.874Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:19.874783Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:19.874Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:19.874783Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:19.875Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:38:19.964Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=185ac7c1-0764-4636-9e36-573e231f6a9c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:19.963864Z InvocationID=6f408030-e58f-458e-9f18-d9b90e0ffce6 LogTimestamp=2021-05-28T00:38:19.963903Z Entering 2021-05-28T00:38:19.964Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=185ac7c1-0764-4636-9e36-573e231f6a9c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:19.963864Z InvocationID=6f408030-e58f-458e-9f18-d9b90e0ffce6 LogTimestamp=2021-05-28T00:38:19.963903Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:38:20.075Z [qtp580718781-16] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:38:19.964Z InvocationID=a9848f71-25a1-4b63-a2fe-77de3e7a4e2f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@15e58f01', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 110ms 2021-05-28T00:38:20.076Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=112 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:38:19.964Z InvocationID=a9848f71-25a1-4b63-a2fe-77de3e7a4e2f ErrorCode=409 LogTimestamp=2021-05-28T00:38:20.076585Z ResponseDesc=Conflict Exiting. 2021-05-28T00:38:20.077Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:38:20.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efd2229b-33d1-4e0e-83fe-9a953ac23fbe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:20.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efd2229b-33d1-4e0e-83fe-9a953ac23fbe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1da7b8c2 2021-05-28T00:38:20.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efd2229b-33d1-4e0e-83fe-9a953ac23fbe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:20.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efd2229b-33d1-4e0e-83fe-9a953ac23fbe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:20.260Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:20.260797Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:20.261Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:20.260797Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:20.261Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:38:20.277Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1f3cb2dc-6555-4595-8d8b-2c2d3c46acae InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:20.277191Z InvocationID=b1a52174-0cd6-4b25-bd42-8be4f0bd4d6f LogTimestamp=2021-05-28T00:38:20.277243Z Entering 2021-05-28T00:38:20.277Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1f3cb2dc-6555-4595-8d8b-2c2d3c46acae InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:20.277191Z InvocationID=b1a52174-0cd6-4b25-bd42-8be4f0bd4d6f LogTimestamp=2021-05-28T00:38:20.277243Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:38:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:38:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:38:20.567Z [qtp580718781-98] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:38:20.277Z InvocationID=4e7b63af-4399-4e72-8870-7ee0eb97b5a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@15e58f01', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 288ms 2021-05-28T00:38:20.567Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=290 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:38:20.277Z InvocationID=4e7b63af-4399-4e72-8870-7ee0eb97b5a6 ErrorCode=409 LogTimestamp=2021-05-28T00:38:20.567682Z ResponseDesc=Conflict Exiting. 2021-05-28T00:38:20.568Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:38:20.873Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=89c7e81e-7e77-4ef1-aa95-ca6c6b3e34f4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:20.873171Z InvocationID=5d7f8dcc-91df-4ad2-b62a-3d2a664daf35 LogTimestamp=2021-05-28T00:38:20.873220Z Entering 2021-05-28T00:38:20.873Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=89c7e81e-7e77-4ef1-aa95-ca6c6b3e34f4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:20.873171Z InvocationID=5d7f8dcc-91df-4ad2-b62a-3d2a664daf35 LogTimestamp=2021-05-28T00:38:20.873220Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:38:20.967Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=94 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:38:20.873Z InvocationID=51815970-9ac5-4967-8474-95e241e23c4e LogTimestamp=2021-05-28T00:38:20.967667Z ResponseDesc=OK Exiting. 2021-05-28T00:38:20.968Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:38:21.084Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:21.083948Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:21.084Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:21.083948Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:21.084Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:38:21.173Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5f07bf25-69ff-4dea-8c30-4271b406b2e3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:21.173378Z InvocationID=656f020a-4a5b-4f3e-93ee-97502c467833 LogTimestamp=2021-05-28T00:38:21.173463Z Entering 2021-05-28T00:38:21.173Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5f07bf25-69ff-4dea-8c30-4271b406b2e3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:21.173378Z InvocationID=656f020a-4a5b-4f3e-93ee-97502c467833 LogTimestamp=2021-05-28T00:38:21.173463Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:21.269Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:21.174Z InvocationID=b404ccb3-2023-4bd5-9c5e-e398ed716007 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:38:23.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edab87a3-a3e1-4e2b-aa17-86e46906c424 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:23.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edab87a3-a3e1-4e2b-aa17-86e46906c424 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11ec3e4d 2021-05-28T00:38:23.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edab87a3-a3e1-4e2b-aa17-86e46906c424 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:23.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edab87a3-a3e1-4e2b-aa17-86e46906c424 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:23.859Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2684 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:21.174Z InvocationID=b404ccb3-2023-4bd5-9c5e-e398ed716007 LogTimestamp=2021-05-28T00:38:23.858984Z ResponseDesc=Created Exiting. 2021-05-28T00:38:23.859Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:24.057Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:24.057015Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:24.057Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:24.057015Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:24.057Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:38:24.170Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4faa83ec-6968-4f8d-959d-4685613cf8c8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:24.169998Z InvocationID=47fe4a96-65c0-439a-b647-ac814c6291a1 LogTimestamp=2021-05-28T00:38:24.170071Z Entering 2021-05-28T00:38:24.170Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4faa83ec-6968-4f8d-959d-4685613cf8c8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:24.169998Z InvocationID=47fe4a96-65c0-439a-b647-ac814c6291a1 LogTimestamp=2021-05-28T00:38:24.170071Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:24.274Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:24.170Z InvocationID=c381931d-b4bd-4cce-a8a2-1211d16873c6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:38:26.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1620934-39d5-49a8-b2cf-3783d58dc46a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:26.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1620934-39d5-49a8-b2cf-3783d58dc46a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ddeeb51 2021-05-28T00:38:26.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1620934-39d5-49a8-b2cf-3783d58dc46a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:26.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1620934-39d5-49a8-b2cf-3783d58dc46a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:26.461Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2290 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:24.170Z InvocationID=c381931d-b4bd-4cce-a8a2-1211d16873c6 LogTimestamp=2021-05-28T00:38:26.460716Z ResponseDesc=Created Exiting. 2021-05-28T00:38:26.461Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:26.557Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:26.557645Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:26.557Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:26.557645Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:26.558Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:38:26.568Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a53b20c0-82a3-4ccc-b35c-3d118fd310b0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:26.568231Z InvocationID=993e5b58-d9e0-4ee3-b7b2-01375401b80e LogTimestamp=2021-05-28T00:38:26.568279Z Entering 2021-05-28T00:38:26.568Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a53b20c0-82a3-4ccc-b35c-3d118fd310b0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:26.568231Z InvocationID=993e5b58-d9e0-4ee3-b7b2-01375401b80e LogTimestamp=2021-05-28T00:38:26.568279Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:26.668Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:26.568Z InvocationID=4cfec898-b208-42f7-aedb-37142b904ba3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:38:28.771Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2203 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:26.568Z InvocationID=4cfec898-b208-42f7-aedb-37142b904ba3 LogTimestamp=2021-05-28T00:38:28.771252Z ResponseDesc=Created Exiting. 2021-05-28T00:38:28.771Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:28.861Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:28.861366Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:28.861Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:28.861366Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:28.861Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:38:28.871Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=58c98515-944a-4b64-904d-cd5f33c3c178 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:28.871677Z InvocationID=8001175c-db7d-4d13-9e6e-945375434235 LogTimestamp=2021-05-28T00:38:28.871714Z Entering 2021-05-28T00:38:28.871Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=58c98515-944a-4b64-904d-cd5f33c3c178 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:28.871677Z InvocationID=8001175c-db7d-4d13-9e6e-945375434235 LogTimestamp=2021-05-28T00:38:28.871714Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:28.962Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:28.872Z InvocationID=05562df4-160f-455e-87a7-a9305e8a7b09 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:38:29.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e44c8a8e-ef03-4d53-8ce1-b42476173108 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:29.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e44c8a8e-ef03-4d53-8ce1-b42476173108 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5739770a 2021-05-28T00:38:29.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e44c8a8e-ef03-4d53-8ce1-b42476173108 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:29.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e44c8a8e-ef03-4d53-8ce1-b42476173108 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:31.468Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2596 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:28.872Z InvocationID=05562df4-160f-455e-87a7-a9305e8a7b09 LogTimestamp=2021-05-28T00:38:31.468577Z ResponseDesc=Created Exiting. 2021-05-28T00:38:31.469Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:31.657Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:31.656946Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:31.657Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:31.656946Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:31.657Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:38:31.763Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f408fe55-fbce-44b3-bb0b-370808bf9ce7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:31.762857Z InvocationID=7989b6d3-74a3-48db-875d-a4da2c2f3275 LogTimestamp=2021-05-28T00:38:31.763228Z Entering 2021-05-28T00:38:31.763Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f408fe55-fbce-44b3-bb0b-370808bf9ce7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:31.762857Z InvocationID=7989b6d3-74a3-48db-875d-a4da2c2f3275 LogTimestamp=2021-05-28T00:38:31.763228Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:31.860Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:31.763Z InvocationID=095a2439-d174-4487-b900-7a13b682b584 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:38:31.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b721f4-8693-4898-95a8-42e1a4a5fa81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:31.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b721f4-8693-4898-95a8-42e1a4a5fa81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7778a487 2021-05-28T00:38:31.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b721f4-8693-4898-95a8-42e1a4a5fa81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:31.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b721f4-8693-4898-95a8-42e1a4a5fa81 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:33.874Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2111 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:31.763Z InvocationID=095a2439-d174-4487-b900-7a13b682b584 LogTimestamp=2021-05-28T00:38:33.874337Z ResponseDesc=Created Exiting. 2021-05-28T00:38:33.874Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:33.973Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:33.973806Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:33.973Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:33.973806Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:33.974Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:38:34.073Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1d793893-e92d-45f3-8d58-7f74426eca25 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:34.073010Z InvocationID=dc5f373a-51ef-4622-9087-a5b6c7de4469 LogTimestamp=2021-05-28T00:38:34.073080Z Entering 2021-05-28T00:38:34.073Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1d793893-e92d-45f3-8d58-7f74426eca25 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:34.073010Z InvocationID=dc5f373a-51ef-4622-9087-a5b6c7de4469 LogTimestamp=2021-05-28T00:38:34.073080Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:34.162Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:34.073Z InvocationID=064fc662-7be9-4775-846c-869c856c571f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:38:34.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f184e4-110c-4e3d-acdb-224a2d00b04d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:34.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f184e4-110c-4e3d-acdb-224a2d00b04d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@796f273 2021-05-28T00:38:34.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f184e4-110c-4e3d-acdb-224a2d00b04d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:34.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f184e4-110c-4e3d-acdb-224a2d00b04d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:38:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:38:36.267Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2194 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:34.073Z InvocationID=064fc662-7be9-4775-846c-869c856c571f LogTimestamp=2021-05-28T00:38:36.267328Z ResponseDesc=Created Exiting. 2021-05-28T00:38:36.267Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:36.369Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:36.369702Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:36.369Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:36.369702Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:36.370Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:38:36.381Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d85dca01-86cb-4bce-9d2b-37d9eec0946d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:36.380919Z InvocationID=29309b93-3d25-4c56-9166-c18cf0c8e6e5 LogTimestamp=2021-05-28T00:38:36.380957Z Entering 2021-05-28T00:38:36.381Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d85dca01-86cb-4bce-9d2b-37d9eec0946d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:36.380919Z InvocationID=29309b93-3d25-4c56-9166-c18cf0c8e6e5 LogTimestamp=2021-05-28T00:38:36.380957Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:36.459Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:36.381Z InvocationID=b69249d3-dfaf-42d4-8549-48af8c861f64 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:38:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:38:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:38:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:38:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76c39eab-0cc2-47cd-9f7f-a66b5a217f51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76c39eab-0cc2-47cd-9f7f-a66b5a217f51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3da815b3 2021-05-28T00:38:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76c39eab-0cc2-47cd-9f7f-a66b5a217f51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76c39eab-0cc2-47cd-9f7f-a66b5a217f51 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:37.978Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1597 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:36.381Z InvocationID=b69249d3-dfaf-42d4-8549-48af8c861f64 LogTimestamp=2021-05-28T00:38:37.978411Z ResponseDesc=Created Exiting. 2021-05-28T00:38:37.978Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:38.072Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:38.072091Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:38.072Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:38.072091Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:38.072Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:38:38.165Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c0b40308-c104-47db-a753-9107a8ea7305 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:38.164995Z InvocationID=af04e7ad-3edf-4a95-b575-b776bfdbcb05 LogTimestamp=2021-05-28T00:38:38.165046Z Entering 2021-05-28T00:38:38.165Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c0b40308-c104-47db-a753-9107a8ea7305 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:38.164995Z InvocationID=af04e7ad-3edf-4a95-b575-b776bfdbcb05 LogTimestamp=2021-05-28T00:38:38.165046Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:38.179Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:38.165Z InvocationID=315a057b-89ec-43b1-bca0-0a6521093c64 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:38:39.667Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1502 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:38.165Z InvocationID=315a057b-89ec-43b1-bca0-0a6521093c64 LogTimestamp=2021-05-28T00:38:39.667428Z ResponseDesc=Created Exiting. 2021-05-28T00:38:39.667Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:39.688Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:39.688049Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:39.688Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:39.688049Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:39.690Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:38:39.872Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3c6008e7-fe5c-4bc0-a9e1-e23e9c755f3f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:39.872279Z InvocationID=451d2249-2847-4880-b472-70850e96861e LogTimestamp=2021-05-28T00:38:39.872331Z Entering 2021-05-28T00:38:39.872Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3c6008e7-fe5c-4bc0-a9e1-e23e9c755f3f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:39.872279Z InvocationID=451d2249-2847-4880-b472-70850e96861e LogTimestamp=2021-05-28T00:38:39.872331Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:39.968Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:39.872Z InvocationID=9abac18b-1984-4d00-b848-786e2a5b24c7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:38:40.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44b5b040-6b63-4484-b371-514341a78f0c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:40.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44b5b040-6b63-4484-b371-514341a78f0c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2135a42b 2021-05-28T00:38:40.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44b5b040-6b63-4484-b371-514341a78f0c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:40.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44b5b040-6b63-4484-b371-514341a78f0c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:41.866Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1994 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:39.872Z InvocationID=9abac18b-1984-4d00-b848-786e2a5b24c7 LogTimestamp=2021-05-28T00:38:41.866371Z ResponseDesc=Created Exiting. 2021-05-28T00:38:41.866Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:41.980Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:41.980193Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:41.980Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:41.980193Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:41.980Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:38:42.076Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=38224a03-ba73-4828-889f-fc98fd3ff0ff InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:42.076042Z InvocationID=cbf734e0-18c3-4cf1-95bb-0f243b2129a6 LogTimestamp=2021-05-28T00:38:42.076086Z Entering 2021-05-28T00:38:42.076Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=38224a03-ba73-4828-889f-fc98fd3ff0ff InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:42.076042Z InvocationID=cbf734e0-18c3-4cf1-95bb-0f243b2129a6 LogTimestamp=2021-05-28T00:38:42.076086Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:42.169Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:42.076Z InvocationID=601a1194-5659-41ea-a099-1c22e79b539a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:38:43.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aaa41a1-7b31-4b66-b063-9ed3111c4ffe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:43.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aaa41a1-7b31-4b66-b063-9ed3111c4ffe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@135b7016 2021-05-28T00:38:43.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aaa41a1-7b31-4b66-b063-9ed3111c4ffe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:43.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aaa41a1-7b31-4b66-b063-9ed3111c4ffe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:43.980Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1904 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:42.076Z InvocationID=601a1194-5659-41ea-a099-1c22e79b539a LogTimestamp=2021-05-28T00:38:43.980098Z ResponseDesc=Created Exiting. 2021-05-28T00:38:43.980Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:44.169Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:44.169306Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:44.169Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:44.169306Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:44.170Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:38:44.266Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=73a5d626-4d12-4d7e-880e-80fd767674c3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:44.266293Z InvocationID=f0622ce3-edab-4fc9-bb85-5be007e887ea LogTimestamp=2021-05-28T00:38:44.266354Z Entering 2021-05-28T00:38:44.266Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=73a5d626-4d12-4d7e-880e-80fd767674c3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:44.266293Z InvocationID=f0622ce3-edab-4fc9-bb85-5be007e887ea LogTimestamp=2021-05-28T00:38:44.266354Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:44.360Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:44.266Z InvocationID=2d5f8140-5995-43fd-8b5b-68ea5078d118 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:38:46.264Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1998 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:44.266Z InvocationID=2d5f8140-5995-43fd-8b5b-68ea5078d118 LogTimestamp=2021-05-28T00:38:46.264321Z ResponseDesc=Created Exiting. 2021-05-28T00:38:46.265Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:46.370Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:46.370400Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:46.370Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:46.370400Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:46.371Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:38:46.463Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=20c0079d-a5f6-458f-9092-a1a3214c2e59 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:46.463473Z InvocationID=9c72da2f-3b3e-4ec0-89af-4b040f9033a7 LogTimestamp=2021-05-28T00:38:46.463535Z Entering 2021-05-28T00:38:46.463Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=20c0079d-a5f6-458f-9092-a1a3214c2e59 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:46.463473Z InvocationID=9c72da2f-3b3e-4ec0-89af-4b040f9033a7 LogTimestamp=2021-05-28T00:38:46.463535Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:46.472Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:46.464Z InvocationID=c4147188-0098-4b56-a677-d386a042deea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:38:46.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13f493ff-2962-433a-bff0-4b07e63f338b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:46.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13f493ff-2962-433a-bff0-4b07e63f338b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23ee0d64 2021-05-28T00:38:46.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13f493ff-2962-433a-bff0-4b07e63f338b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:46.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13f493ff-2962-433a-bff0-4b07e63f338b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:47.977Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1512 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:46.464Z InvocationID=c4147188-0098-4b56-a677-d386a042deea LogTimestamp=2021-05-28T00:38:47.976915Z ResponseDesc=Created Exiting. 2021-05-28T00:38:47.977Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:48.062Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:48.062464Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:48.062Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:48.062464Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:48.063Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:38:48.073Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d6f1c14a-8320-492f-abd8-7025b255c4ab InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:48.073284Z InvocationID=a7f01b18-2f80-439b-96ca-0ccf8e5b242d LogTimestamp=2021-05-28T00:38:48.073346Z Entering 2021-05-28T00:38:48.073Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d6f1c14a-8320-492f-abd8-7025b255c4ab InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:48.073284Z InvocationID=a7f01b18-2f80-439b-96ca-0ccf8e5b242d LogTimestamp=2021-05-28T00:38:48.073346Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:48.085Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:48.073Z InvocationID=78b559df-9f17-4475-b7ec-a81af85f5e69 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:38:49.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dc66471-f6d9-45e8-855f-7a60a1b9763b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:49.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dc66471-f6d9-45e8-855f-7a60a1b9763b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e168127 2021-05-28T00:38:49.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dc66471-f6d9-45e8-855f-7a60a1b9763b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:49.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dc66471-f6d9-45e8-855f-7a60a1b9763b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:49.680Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1607 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:48.073Z InvocationID=78b559df-9f17-4475-b7ec-a81af85f5e69 LogTimestamp=2021-05-28T00:38:49.680268Z ResponseDesc=Created Exiting. 2021-05-28T00:38:49.680Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:49.772Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:49.772322Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:49.772Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:49.772322Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:49.773Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:38:49.857Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4ebf0bee-5719-4281-b9cb-34ed57235fab InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:49.857126Z InvocationID=1b503ec6-c5ec-4886-9108-b99699068b14 LogTimestamp=2021-05-28T00:38:49.857172Z Entering 2021-05-28T00:38:49.857Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4ebf0bee-5719-4281-b9cb-34ed57235fab InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:49.857126Z InvocationID=1b503ec6-c5ec-4886-9108-b99699068b14 LogTimestamp=2021-05-28T00:38:49.857172Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:49.875Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:49.857Z InvocationID=ef6593de-35e8-44e8-8e7f-528042c25d74 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:38:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:38:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:38:51.378Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1520 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:49.857Z InvocationID=ef6593de-35e8-44e8-8e7f-528042c25d74 LogTimestamp=2021-05-28T00:38:51.377736Z ResponseDesc=Created Exiting. 2021-05-28T00:38:51.378Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:51.472Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:51.472522Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:38:51.472Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:38:51.472522Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:51.473Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:38:51.569Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0826d5ff-0407-4fcb-9197-38f09fae3d5a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:51.569020Z InvocationID=ac461e98-a32a-44cc-a351-a901b55c02f0 LogTimestamp=2021-05-28T00:38:51.569072Z Entering 2021-05-28T00:38:51.569Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0826d5ff-0407-4fcb-9197-38f09fae3d5a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:51.569020Z InvocationID=ac461e98-a32a-44cc-a351-a901b55c02f0 LogTimestamp=2021-05-28T00:38:51.569072Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:38:51.577Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:51.569Z InvocationID=95cdc349-ded3-4fd7-b498-da4cf63a34ee PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:38:52.122Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:38:52.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aa83be1-9627-4d08-8738-b767d380991b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:52.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aa83be1-9627-4d08-8738-b767d380991b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@520e7232 2021-05-28T00:38:52.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aa83be1-9627-4d08-8738-b767d380991b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:52.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3aa83be1-9627-4d08-8738-b767d380991b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:53.459Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1889 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:38:51.569Z InvocationID=95cdc349-ded3-4fd7-b498-da4cf63a34ee LogTimestamp=2021-05-28T00:38:53.458817Z ResponseDesc=Created Exiting. 2021-05-28T00:38:53.459Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:38:53.873Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=827d251f-f802-4a52-8b0e-160735294767 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:53.873198Z InvocationID=b25de506-f48b-4d86-93a5-8dd8b7a5eb57 LogTimestamp=2021-05-28T00:38:53.873252Z Entering 2021-05-28T00:38:53.873Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=827d251f-f802-4a52-8b0e-160735294767 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:53.873198Z InvocationID=b25de506-f48b-4d86-93a5-8dd8b7a5eb57 LogTimestamp=2021-05-28T00:38:53.873252Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:38:53.883Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:38:53.873Z InvocationID=a5a071ce-d440-45e9-a8fe-3479bc3ba256 LogTimestamp=2021-05-28T00:38:53.882837Z ResponseDesc=OK Exiting. 2021-05-28T00:38:53.883Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:38:54.376Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=12a18592-2005-407c-b0d0-f7f29730b8a1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:54.376439Z InvocationID=fe445a4c-f3b0-448b-835b-291c22126b8d LogTimestamp=2021-05-28T00:38:54.376485Z Entering 2021-05-28T00:38:54.376Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=12a18592-2005-407c-b0d0-f7f29730b8a1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:54.376439Z InvocationID=fe445a4c-f3b0-448b-835b-291c22126b8d LogTimestamp=2021-05-28T00:38:54.376485Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:38:54.575Z [qtp580718781-11] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:38:54.376Z InvocationID=39493aa2-0ed5-49fa-8e46-01e96f8ac26c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 197063254 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n197063254 100% \n 2021-05-28T00:38:54.582Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=206 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:38:54.376Z InvocationID=39493aa2-0ed5-49fa-8e46-01e96f8ac26c LogTimestamp=2021-05-28T00:38:54.582540Z ResponseDesc=OK Exiting. 2021-05-28T00:38:54.582Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:38:54.765Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4ae19799-f447-4a94-8803-2ecf3ab8a04b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:54.765438Z InvocationID=f7704a9a-03e4-450e-a612-a8819a9c1b8c LogTimestamp=2021-05-28T00:38:54.765477Z Entering 2021-05-28T00:38:54.765Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4ae19799-f447-4a94-8803-2ecf3ab8a04b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:38:54.765438Z InvocationID=f7704a9a-03e4-450e-a612-a8819a9c1b8c LogTimestamp=2021-05-28T00:38:54.765477Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:38:54.860Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:38:54.765Z InvocationID=cbdab7df-ebb7-429e-9839-ad19a02c3c17 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:38:54.869Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:38:54.765Z InvocationID=cbdab7df-ebb7-429e-9839-ad19a02c3c17 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:38:54.870Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:38:54.765Z InvocationID=cbdab7df-ebb7-429e-9839-ad19a02c3c17 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:38:54.870Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=105 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:38:54.765Z InvocationID=cbdab7df-ebb7-429e-9839-ad19a02c3c17 ErrorCode=500 LogTimestamp=2021-05-28T00:38:54.870570Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:38:54.870Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:38:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e53b87-35ff-4e8e-afc1-347913c848a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e53b87-35ff-4e8e-afc1-347913c848a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a9d73a5 2021-05-28T00:38:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e53b87-35ff-4e8e-afc1-347913c848a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e53b87-35ff-4e8e-afc1-347913c848a9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:38:58.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c550520-fe30-4044-9e82-f1659822365c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:38:58.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c550520-fe30-4044-9e82-f1659822365c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@494bc8bf 2021-05-28T00:38:58.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c550520-fe30-4044-9e82-f1659822365c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:38:58.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c550520-fe30-4044-9e82-f1659822365c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:01.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f53e68-f0c7-4828-8f7f-55a0453c6d77 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:01.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f53e68-f0c7-4828-8f7f-55a0453c6d77 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@233e7708 2021-05-28T00:39:01.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f53e68-f0c7-4828-8f7f-55a0453c6d77 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:01.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21f53e68-f0c7-4828-8f7f-55a0453c6d77 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:04.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc149721-a8ae-4b95-99dc-b0e88e84aca4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:04.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc149721-a8ae-4b95-99dc-b0e88e84aca4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2001ebf 2021-05-28T00:39:04.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc149721-a8ae-4b95-99dc-b0e88e84aca4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:04.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc149721-a8ae-4b95-99dc-b0e88e84aca4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:05.062Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6587b3d0-7d37-4533-872d-edbb8627274f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:05.061709Z InvocationID=5b220728-f195-43d0-a0e0-6c8201a15410 LogTimestamp=2021-05-28T00:39:05.061827Z Entering 2021-05-28T00:39:05.062Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6587b3d0-7d37-4533-872d-edbb8627274f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:05.061709Z InvocationID=5b220728-f195-43d0-a0e0-6c8201a15410 LogTimestamp=2021-05-28T00:39:05.061827Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:39:05.188Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:39:05.062Z InvocationID=cf66d68f-21cf-4f6c-bdfd-9c4f2bb84ccc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:39:05.268Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:05.062Z InvocationID=cf66d68f-21cf-4f6c-bdfd-9c4f2bb84ccc ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:39:05.269Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:05.062Z InvocationID=cf66d68f-21cf-4f6c-bdfd-9c4f2bb84ccc ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:39:05.270Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=207 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:39:05.062Z InvocationID=cf66d68f-21cf-4f6c-bdfd-9c4f2bb84ccc ErrorCode=500 LogTimestamp=2021-05-28T00:39:05.269912Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:39:05.270Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:39:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:39:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:39:07.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05c31719-8f94-404c-a8c8-4f38b765c2bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:07.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05c31719-8f94-404c-a8c8-4f38b765c2bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73714501 2021-05-28T00:39:07.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05c31719-8f94-404c-a8c8-4f38b765c2bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:07.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05c31719-8f94-404c-a8c8-4f38b765c2bf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:08.142Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ys1940mhbdxe1fovmzu7kl6rm0 2021-05-28T00:39:08.143Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0omv5qa2pxvn4ts00v42k92px1 2021-05-28T00:39:10.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a204c47-c258-4fef-8e51-66e4a636c14e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:10.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a204c47-c258-4fef-8e51-66e4a636c14e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64e875e0 2021-05-28T00:39:10.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a204c47-c258-4fef-8e51-66e4a636c14e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:10.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a204c47-c258-4fef-8e51-66e4a636c14e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:13.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7d26eaa-cd78-4367-9bf3-16cf6e9ea696 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:13.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7d26eaa-cd78-4367-9bf3-16cf6e9ea696 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@343ed842 2021-05-28T00:39:13.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7d26eaa-cd78-4367-9bf3-16cf6e9ea696 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:13.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7d26eaa-cd78-4367-9bf3-16cf6e9ea696 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:15.570Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2516136a-11f2-425b-9477-cf1e20fad376 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:15.569585Z InvocationID=635131fc-4189-4910-94de-856a6eace515 LogTimestamp=2021-05-28T00:39:15.569671Z Entering 2021-05-28T00:39:15.570Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2516136a-11f2-425b-9477-cf1e20fad376 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:15.569585Z InvocationID=635131fc-4189-4910-94de-856a6eace515 LogTimestamp=2021-05-28T00:39:15.569671Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:39:15.660Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:39:15.570Z InvocationID=5516c828-e647-47cc-a7a4-098f7d0f9ee6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:39:15.665Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:15.570Z InvocationID=5516c828-e647-47cc-a7a4-098f7d0f9ee6 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:39:15.666Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:15.570Z InvocationID=5516c828-e647-47cc-a7a4-098f7d0f9ee6 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:39:15.666Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=96 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:39:15.570Z InvocationID=5516c828-e647-47cc-a7a4-098f7d0f9ee6 ErrorCode=500 LogTimestamp=2021-05-28T00:39:15.666615Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:39:15.667Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:39:16.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f0ed3e3-ea3e-4dd3-a91a-f72c9b99f596 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:16.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f0ed3e3-ea3e-4dd3-a91a-f72c9b99f596 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36c9dd9c 2021-05-28T00:39:16.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f0ed3e3-ea3e-4dd3-a91a-f72c9b99f596 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:16.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f0ed3e3-ea3e-4dd3-a91a-f72c9b99f596 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33341710-16f9-44ab-ac74-e64d86cf7ec2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33341710-16f9-44ab-ac74-e64d86cf7ec2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f69df09 2021-05-28T00:39:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33341710-16f9-44ab-ac74-e64d86cf7ec2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:19.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33341710-16f9-44ab-ac74-e64d86cf7ec2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:39:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:39:22.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2300a614-4af7-4b63-a305-76d81a3301f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:22.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2300a614-4af7-4b63-a305-76d81a3301f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6889e471 2021-05-28T00:39:22.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2300a614-4af7-4b63-a305-76d81a3301f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:22.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2300a614-4af7-4b63-a305-76d81a3301f4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:25.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=098f3d77-5990-4c56-b7fd-a33c54d84c31 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:25.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=098f3d77-5990-4c56-b7fd-a33c54d84c31 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a37cdea 2021-05-28T00:39:25.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=098f3d77-5990-4c56-b7fd-a33c54d84c31 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:25.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=098f3d77-5990-4c56-b7fd-a33c54d84c31 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:39:25.764Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2537c634-10f7-43b3-8ca2-e0cb1a50fda9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:25.763738Z InvocationID=32b42f2a-d4cf-4923-8628-91e3b12c0fea LogTimestamp=2021-05-28T00:39:25.763843Z Entering 2021-05-28T00:39:25.764Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2537c634-10f7-43b3-8ca2-e0cb1a50fda9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:25.763738Z InvocationID=32b42f2a-d4cf-4923-8628-91e3b12c0fea LogTimestamp=2021-05-28T00:39:25.763843Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:39:25.908Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:39:25.764Z InvocationID=7cf7ae3d-9bce-4077-abcb-471d9b172f25 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:39:26.018Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:25.764Z InvocationID=7cf7ae3d-9bce-4077-abcb-471d9b172f25 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:39:26.018Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:25.764Z InvocationID=7cf7ae3d-9bce-4077-abcb-471d9b172f25 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:39:26.019Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=255 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:39:25.764Z InvocationID=7cf7ae3d-9bce-4077-abcb-471d9b172f25 ErrorCode=500 LogTimestamp=2021-05-28T00:39:26.019565Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:39:26.020Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:39:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31346f35-3ce3-416b-9c7a-6332544a667d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31346f35-3ce3-416b-9c7a-6332544a667d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52eac00d 2021-05-28T00:39:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31346f35-3ce3-416b-9c7a-6332544a667d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31346f35-3ce3-416b-9c7a-6332544a667d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:31.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94d0388e-d534-47ff-ac11-e1dffcb4a4e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:31.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94d0388e-d534-47ff-ac11-e1dffcb4a4e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@537accce 2021-05-28T00:39:31.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94d0388e-d534-47ff-ac11-e1dffcb4a4e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:31.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94d0388e-d534-47ff-ac11-e1dffcb4a4e6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:34.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba60345-6808-4c5b-b003-f08d76a00515 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:34.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba60345-6808-4c5b-b003-f08d76a00515 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@747700bc 2021-05-28T00:39:34.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba60345-6808-4c5b-b003-f08d76a00515 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:34.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba60345-6808-4c5b-b003-f08d76a00515 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:39:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:39:36.819Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a01d449a-ce20-43b9-bb51-4aa1a0e0c4a8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:36.818791Z InvocationID=dff66f19-dcfe-41e8-89f0-318b4de9e984 LogTimestamp=2021-05-28T00:39:36.818923Z Entering 2021-05-28T00:39:36.819Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a01d449a-ce20-43b9-bb51-4aa1a0e0c4a8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:36.818791Z InvocationID=dff66f19-dcfe-41e8-89f0-318b4de9e984 LogTimestamp=2021-05-28T00:39:36.818923Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:39:36.927Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:39:36.819Z InvocationID=e716907d-bb7a-48b7-af17-1b3c2f737ba1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:39:36.970Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:36.819Z InvocationID=e716907d-bb7a-48b7-af17-1b3c2f737ba1 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:39:36.970Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:36.819Z InvocationID=e716907d-bb7a-48b7-af17-1b3c2f737ba1 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:39:36.971Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=152 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:39:36.819Z InvocationID=e716907d-bb7a-48b7-af17-1b3c2f737ba1 ErrorCode=500 LogTimestamp=2021-05-28T00:39:36.971323Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:39:36.971Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:39:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:39:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:39:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:39:38.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c00a94e4-c9ac-4e18-b1e5-8b48da5d54f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:38.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c00a94e4-c9ac-4e18-b1e5-8b48da5d54f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@603bdb45 2021-05-28T00:39:38.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c00a94e4-c9ac-4e18-b1e5-8b48da5d54f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:38.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c00a94e4-c9ac-4e18-b1e5-8b48da5d54f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:40.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c341f7-a47e-4549-97b7-d53d8fa6ed69 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:40.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c341f7-a47e-4549-97b7-d53d8fa6ed69 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64d585e7 2021-05-28T00:39:40.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c341f7-a47e-4549-97b7-d53d8fa6ed69 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:40.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4c341f7-a47e-4549-97b7-d53d8fa6ed69 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:43.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e7bfd7-efc0-45b0-9562-c164af4eb449 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:43.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e7bfd7-efc0-45b0-9562-c164af4eb449 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@614c390f 2021-05-28T00:39:43.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e7bfd7-efc0-45b0-9562-c164af4eb449 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:43.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19e7bfd7-efc0-45b0-9562-c164af4eb449 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7abe9121-b42a-42ea-a0f2-4fbb34f7e8ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7abe9121-b42a-42ea-a0f2-4fbb34f7e8ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d72a77a 2021-05-28T00:39:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7abe9121-b42a-42ea-a0f2-4fbb34f7e8ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7abe9121-b42a-42ea-a0f2-4fbb34f7e8ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:47.065Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bfab1c63-bd9f-4079-973b-6a01236e5d9d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:47.064915Z InvocationID=652f7f13-1991-4711-a13e-d03003e94387 LogTimestamp=2021-05-28T00:39:47.064962Z Entering 2021-05-28T00:39:47.066Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bfab1c63-bd9f-4079-973b-6a01236e5d9d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:47.064915Z InvocationID=652f7f13-1991-4711-a13e-d03003e94387 LogTimestamp=2021-05-28T00:39:47.064962Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:39:47.172Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:39:47.066Z InvocationID=27154763-06b3-4844-a55f-b8322b920ee6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:39:47.184Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:47.066Z InvocationID=27154763-06b3-4844-a55f-b8322b920ee6 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:39:47.184Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:47.066Z InvocationID=27154763-06b3-4844-a55f-b8322b920ee6 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:39:47.185Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=119 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:39:47.066Z InvocationID=27154763-06b3-4844-a55f-b8322b920ee6 ErrorCode=500 LogTimestamp=2021-05-28T00:39:47.185231Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:39:47.185Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:39:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf5076f7-965c-4d2c-851c-41a6b190b741 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf5076f7-965c-4d2c-851c-41a6b190b741 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13adb062 2021-05-28T00:39:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf5076f7-965c-4d2c-851c-41a6b190b741 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:49.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf5076f7-965c-4d2c-851c-41a6b190b741 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:39:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:39:52.251Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:39:53.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1ecd3c-9da2-49f3-8c16-cd35fc2e933c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:53.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1ecd3c-9da2-49f3-8c16-cd35fc2e933c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b1ff54d 2021-05-28T00:39:53.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1ecd3c-9da2-49f3-8c16-cd35fc2e933c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:53.390Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1ecd3c-9da2-49f3-8c16-cd35fc2e933c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:56.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdfb7b10-e436-4284-b825-a0bc1a7d9614 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:56.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdfb7b10-e436-4284-b825-a0bc1a7d9614 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b9b3b60 2021-05-28T00:39:56.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdfb7b10-e436-4284-b825-a0bc1a7d9614 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:56.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdfb7b10-e436-4284-b825-a0bc1a7d9614 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:39:57.273Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=81e60988-67f7-40ea-bc3a-50ef25f19749 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:57.273154Z InvocationID=b6448192-b6bb-4420-a0a9-4b3673846a20 LogTimestamp=2021-05-28T00:39:57.273282Z Entering 2021-05-28T00:39:57.273Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=81e60988-67f7-40ea-bc3a-50ef25f19749 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:39:57.273154Z InvocationID=b6448192-b6bb-4420-a0a9-4b3673846a20 LogTimestamp=2021-05-28T00:39:57.273282Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:39:57.370Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:39:57.274Z InvocationID=a6f657a5-e1ba-428c-8ae2-7e43af93e0f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:39:57.455Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:57.274Z InvocationID=a6f657a5-e1ba-428c-8ae2-7e43af93e0f6 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:39:57.456Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:39:57.274Z InvocationID=a6f657a5-e1ba-428c-8ae2-7e43af93e0f6 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:39:57.456Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=182 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:39:57.274Z InvocationID=a6f657a5-e1ba-428c-8ae2-7e43af93e0f6 ErrorCode=500 LogTimestamp=2021-05-28T00:39:57.456705Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:39:57.457Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:39:59.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bcdbf5c-734c-46a3-bd6a-076648baf3c8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:39:59.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bcdbf5c-734c-46a3-bd6a-076648baf3c8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@636a0e1e 2021-05-28T00:39:59.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bcdbf5c-734c-46a3-bd6a-076648baf3c8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:39:59.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bcdbf5c-734c-46a3-bd6a-076648baf3c8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0af83aa-1cf9-4507-b056-e5117495cc71 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0af83aa-1cf9-4507-b056-e5117495cc71 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d3a9cb1 2021-05-28T00:40:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0af83aa-1cf9-4507-b056-e5117495cc71 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0af83aa-1cf9-4507-b056-e5117495cc71 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:04.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4f6af3-e545-434b-aa40-96bbf843f90b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:04.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4f6af3-e545-434b-aa40-96bbf843f90b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@392ba7c6 2021-05-28T00:40:04.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4f6af3-e545-434b-aa40-96bbf843f90b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:04.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4f6af3-e545-434b-aa40-96bbf843f90b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:40:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:40:07.482Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6a432f21-d825-4a2d-a7b0-59242afb25e1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:07.482586Z InvocationID=71abb3d9-0497-4327-8428-9834b858bc16 LogTimestamp=2021-05-28T00:40:07.482632Z Entering 2021-05-28T00:40:07.482Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6a432f21-d825-4a2d-a7b0-59242afb25e1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:07.482586Z InvocationID=71abb3d9-0497-4327-8428-9834b858bc16 LogTimestamp=2021-05-28T00:40:07.482632Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:40:07.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f3e67c-f883-4c38-ae67-15e6c4017523 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:07.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f3e67c-f883-4c38-ae67-15e6c4017523 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3be9dbb8 2021-05-28T00:40:07.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f3e67c-f883-4c38-ae67-15e6c4017523 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:07.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f3e67c-f883-4c38-ae67-15e6c4017523 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:07.588Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:40:07.483Z InvocationID=1e94bfaa-eff4-446c-a667-4821dd48ea53 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:40:07.606Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:40:07.483Z InvocationID=1e94bfaa-eff4-446c-a667-4821dd48ea53 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:40:07.606Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:40:07.483Z InvocationID=1e94bfaa-eff4-446c-a667-4821dd48ea53 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:40:07.607Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=124 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:40:07.483Z InvocationID=1e94bfaa-eff4-446c-a667-4821dd48ea53 ErrorCode=500 LogTimestamp=2021-05-28T00:40:07.607500Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:40:07.608Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:40:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118809ee-5aa0-471e-b331-587f708ef758 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118809ee-5aa0-471e-b331-587f708ef758 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8402825 2021-05-28T00:40:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118809ee-5aa0-471e-b331-587f708ef758 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:10.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118809ee-5aa0-471e-b331-587f708ef758 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:15.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cd428e0-b7c9-4e00-9c66-c45b7b93d226 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:15.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cd428e0-b7c9-4e00-9c66-c45b7b93d226 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41d6e288 2021-05-28T00:40:15.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cd428e0-b7c9-4e00-9c66-c45b7b93d226 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:15.790Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cd428e0-b7c9-4e00-9c66-c45b7b93d226 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:17.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96de389c-17be-49ca-8719-35956deb31e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:17.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96de389c-17be-49ca-8719-35956deb31e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25a768e3 2021-05-28T00:40:17.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96de389c-17be-49ca-8719-35956deb31e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:17.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96de389c-17be-49ca-8719-35956deb31e4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:17.684Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c405beb1-c609-4c28-b9e4-70a3b033a6ac InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:17.684637Z InvocationID=a69af653-7f20-4da0-a861-d720385c2e3c LogTimestamp=2021-05-28T00:40:17.684702Z Entering 2021-05-28T00:40:17.685Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c405beb1-c609-4c28-b9e4-70a3b033a6ac InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:17.684637Z InvocationID=a69af653-7f20-4da0-a861-d720385c2e3c LogTimestamp=2021-05-28T00:40:17.684702Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:40:17.983Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:40:17.685Z InvocationID=6a88c9c4-7729-466a-b407-b3df0326e5b7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:40:18.071Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:40:17.685Z InvocationID=6a88c9c4-7729-466a-b407-b3df0326e5b7 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:40:18.072Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:40:17.685Z InvocationID=6a88c9c4-7729-466a-b407-b3df0326e5b7 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:40:18.072Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=387 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:40:17.685Z InvocationID=6a88c9c4-7729-466a-b407-b3df0326e5b7 ErrorCode=500 LogTimestamp=2021-05-28T00:40:18.072738Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:40:18.073Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:40:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b411a-5cdd-4dca-a9a9-ef3905b2703a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b411a-5cdd-4dca-a9a9-ef3905b2703a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@506643f4 2021-05-28T00:40:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b411a-5cdd-4dca-a9a9-ef3905b2703a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b411a-5cdd-4dca-a9a9-ef3905b2703a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:40:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:40:23.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c033059d-e99f-4310-a39f-d8991b5db2fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:23.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c033059d-e99f-4310-a39f-d8991b5db2fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@596e58dd 2021-05-28T00:40:23.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c033059d-e99f-4310-a39f-d8991b5db2fc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:23.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c033059d-e99f-4310-a39f-d8991b5db2fc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:25.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aee70866-de95-4c41-9dfd-c7a3c915db25 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:25.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aee70866-de95-4c41-9dfd-c7a3c915db25 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f5a23da 2021-05-28T00:40:25.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aee70866-de95-4c41-9dfd-c7a3c915db25 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:25.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aee70866-de95-4c41-9dfd-c7a3c915db25 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:28.272Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=44c1adf1-dc6d-49f7-be32-0b44294fb932 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:28.272550Z InvocationID=b6dc1d90-f976-4b8b-83ca-9059f97bf4a9 LogTimestamp=2021-05-28T00:40:28.272632Z Entering 2021-05-28T00:40:28.273Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=44c1adf1-dc6d-49f7-be32-0b44294fb932 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:28.272550Z InvocationID=b6dc1d90-f976-4b8b-83ca-9059f97bf4a9 LogTimestamp=2021-05-28T00:40:28.272632Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:40:28.368Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:40:28.273Z InvocationID=81bf9cad-8949-4393-a517-01b2dea099a2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:40:28.398Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:40:28.273Z InvocationID=81bf9cad-8949-4393-a517-01b2dea099a2 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:40:28.399Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:40:28.273Z InvocationID=81bf9cad-8949-4393-a517-01b2dea099a2 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:40:28.400Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=126 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:40:28.273Z InvocationID=81bf9cad-8949-4393-a517-01b2dea099a2 ErrorCode=500 LogTimestamp=2021-05-28T00:40:28.399813Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:40:28.400Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:40:28.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dde9edb-6a82-4a08-840e-2547e22e05b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:28.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dde9edb-6a82-4a08-840e-2547e22e05b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31a4b6a2 2021-05-28T00:40:28.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dde9edb-6a82-4a08-840e-2547e22e05b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:28.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dde9edb-6a82-4a08-840e-2547e22e05b3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:31.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7081afd6-06b4-4c80-bc79-760cc24c86dd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:31.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7081afd6-06b4-4c80-bc79-760cc24c86dd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@533f2a81 2021-05-28T00:40:31.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7081afd6-06b4-4c80-bc79-760cc24c86dd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:31.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7081afd6-06b4-4c80-bc79-760cc24c86dd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:35.397Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8295d3-fa58-4a26-a649-10dca773c9cc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:35.397Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8295d3-fa58-4a26-a649-10dca773c9cc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@783c288b 2021-05-28T00:40:35.397Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8295d3-fa58-4a26-a649-10dca773c9cc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:35.397Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8295d3-fa58-4a26-a649-10dca773c9cc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:40:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:40:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:40:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:40:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:40:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:40:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:40:38.561Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=68c19fdf-ca03-4f99-86a5-85f4b1b1fabc InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:38.560606Z InvocationID=9d0e5d2d-94be-4bf1-be16-854233e5092d LogTimestamp=2021-05-28T00:40:38.560748Z Entering 2021-05-28T00:40:38.561Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=68c19fdf-ca03-4f99-86a5-85f4b1b1fabc InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.179 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.179 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:40:38.560606Z InvocationID=9d0e5d2d-94be-4bf1-be16-854233e5092d LogTimestamp=2021-05-28T00:40:38.560748Z localAddr=10.242.102.221 remoteAddr=10.242.170.179 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:40:38.562Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:40:38.562Z InvocationID=aa255228-b5da-48f5-b13b-73c70f9a1be9 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:40:38.564Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:40:38.563Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.179 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:40:38.562Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=aa255228-b5da-48f5-b13b-73c70f9a1be9 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@1c3aff42 2021-05-28T00:40:38.564Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:40:38.563Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.179 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:40:38.562Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=aa255228-b5da-48f5-b13b-73c70f9a1be9 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@1c3aff42 2021-05-28T00:40:38.565Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:40:38.565045Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:40:38.565Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:40:38.565045Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:40:38.565Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:40:38.826Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=232e9cf2-2106-47a6-8670-c89916d209ee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:38.826Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=232e9cf2-2106-47a6-8670-c89916d209ee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68690a5 2021-05-28T00:40:38.826Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=232e9cf2-2106-47a6-8670-c89916d209ee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:38.826Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=232e9cf2-2106-47a6-8670-c89916d209ee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:40.667Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:40:40.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edf613a2-c875-4875-a8b2-e96a72ac5f9a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:40.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edf613a2-c875-4875-a8b2-e96a72ac5f9a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12833771 2021-05-28T00:40:40.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edf613a2-c875-4875-a8b2-e96a72ac5f9a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:40.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edf613a2-c875-4875-a8b2-e96a72ac5f9a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:40.983Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=edf613a2-c875-4875-a8b2-e96a72ac5f9a ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:40:40.983Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=edf613a2-c875-4875-a8b2-e96a72ac5f9a ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:40:43.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64c829d0-d723-4714-8968-8288c13f432b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:43.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64c829d0-d723-4714-8968-8288c13f432b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38c26c82 2021-05-28T00:40:43.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64c829d0-d723-4714-8968-8288c13f432b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:43.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64c829d0-d723-4714-8968-8288c13f432b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:43.986Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=64c829d0-d723-4714-8968-8288c13f432b ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:40:43.986Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=64c829d0-d723-4714-8968-8288c13f432b ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:40:46.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e3fd9e-fad7-47a8-abbe-41e807156112 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:46.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e3fd9e-fad7-47a8-abbe-41e807156112 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21237361 2021-05-28T00:40:46.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e3fd9e-fad7-47a8-abbe-41e807156112 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:46.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e3fd9e-fad7-47a8-abbe-41e807156112 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:50.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8685e4c-963b-402a-9e5b-11ea8a723d31 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:40:50.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8685e4c-963b-402a-9e5b-11ea8a723d31 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1392cc58 2021-05-28T00:40:50.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8685e4c-963b-402a-9e5b-11ea8a723d31 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:40:50.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8685e4c-963b-402a-9e5b-11ea8a723d31 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:40:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:40:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:40:52.159Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:41:03.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51651be0-f72c-4fe7-95ad-2cd214e9e8c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:03.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51651be0-f72c-4fe7-95ad-2cd214e9e8c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@182159e4 2021-05-28T00:41:03.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51651be0-f72c-4fe7-95ad-2cd214e9e8c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:03.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51651be0-f72c-4fe7-95ad-2cd214e9e8c7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:05.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa4eff11-c5c9-4aa8-bdc3-8816c7dd656a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:05.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa4eff11-c5c9-4aa8-bdc3-8816c7dd656a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@332c801e 2021-05-28T00:41:05.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa4eff11-c5c9-4aa8-bdc3-8816c7dd656a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:05.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa4eff11-c5c9-4aa8-bdc3-8816c7dd656a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:41:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:41:07.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0225943-6103-4411-b07d-b06af8710065 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:07.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0225943-6103-4411-b07d-b06af8710065 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9ae5561 2021-05-28T00:41:07.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0225943-6103-4411-b07d-b06af8710065 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:07.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0225943-6103-4411-b07d-b06af8710065 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:08.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36e07296-60ac-422d-9d0c-e9e4e84b6d05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:08.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36e07296-60ac-422d-9d0c-e9e4e84b6d05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c5bb800 2021-05-28T00:41:08.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36e07296-60ac-422d-9d0c-e9e4e84b6d05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:08.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36e07296-60ac-422d-9d0c-e9e4e84b6d05 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:09.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4153aed9-201f-449e-a9e9-b2f2f6e46804 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:09.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4153aed9-201f-449e-a9e9-b2f2f6e46804 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ea73e16 2021-05-28T00:41:09.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4153aed9-201f-449e-a9e9-b2f2f6e46804 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:09.083Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4153aed9-201f-449e-a9e9-b2f2f6e46804 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:10.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8b1105-a338-4944-bdff-3dc7188a9e78 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:10.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8b1105-a338-4944-bdff-3dc7188a9e78 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@495c71ac 2021-05-28T00:41:10.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8b1105-a338-4944-bdff-3dc7188a9e78 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:10.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e8b1105-a338-4944-bdff-3dc7188a9e78 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec07e798-f093-4fc4-b98b-d4efcacae813 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec07e798-f093-4fc4-b98b-d4efcacae813 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32a220bb 2021-05-28T00:41:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec07e798-f093-4fc4-b98b-d4efcacae813 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec07e798-f093-4fc4-b98b-d4efcacae813 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:18.060Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:41:18.072Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:41:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:41:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:41:24.065Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:41:24.066Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:41:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:41:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:41:35.010Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1713b098-cdbf-4273-b24f-4006d9874f78 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:35.010Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1713b098-cdbf-4273-b24f-4006d9874f78 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63681557 2021-05-28T00:41:35.011Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1713b098-cdbf-4273-b24f-4006d9874f78 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:35.011Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1713b098-cdbf-4273-b24f-4006d9874f78 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:41:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:41:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:41:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:41:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:41:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:41:40.078Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:41:40.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cccb82d-4923-4bbe-b411-0c362c9b71be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:40.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cccb82d-4923-4bbe-b411-0c362c9b71be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4498d0a2 2021-05-28T00:41:40.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cccb82d-4923-4bbe-b411-0c362c9b71be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:40.558Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cccb82d-4923-4bbe-b411-0c362c9b71be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:40.561Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0cccb82d-4923-4bbe-b411-0c362c9b71be ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:41:40.562Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=0cccb82d-4923-4bbe-b411-0c362c9b71be ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:41:43.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdaa2397-5d16-4d55-925b-f5c3e3a06614 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:43.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdaa2397-5d16-4d55-925b-f5c3e3a06614 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ee3188c 2021-05-28T00:41:43.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdaa2397-5d16-4d55-925b-f5c3e3a06614 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:43.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdaa2397-5d16-4d55-925b-f5c3e3a06614 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:43.784Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fdaa2397-5d16-4d55-925b-f5c3e3a06614 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:41:43.784Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fdaa2397-5d16-4d55-925b-f5c3e3a06614 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:41:44.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c409d46-b4dd-4bf0-94ba-45b53e250038 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:44.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c409d46-b4dd-4bf0-94ba-45b53e250038 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4de75a13 2021-05-28T00:41:44.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c409d46-b4dd-4bf0-94ba-45b53e250038 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:44.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c409d46-b4dd-4bf0-94ba-45b53e250038 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:45.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d34d3452-ddb6-40e7-bede-bea5ff0eee6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:45.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d34d3452-ddb6-40e7-bede-bea5ff0eee6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e863c55 2021-05-28T00:41:45.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d34d3452-ddb6-40e7-bede-bea5ff0eee6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:45.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d34d3452-ddb6-40e7-bede-bea5ff0eee6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:46.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24905efc-eab7-4277-b70b-3ae288167e32 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:46.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24905efc-eab7-4277-b70b-3ae288167e32 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c19fce7 2021-05-28T00:41:46.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24905efc-eab7-4277-b70b-3ae288167e32 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:46.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24905efc-eab7-4277-b70b-3ae288167e32 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:49.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50d53c2c-1c62-4a0b-9ff1-e583e795700a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:49.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50d53c2c-1c62-4a0b-9ff1-e583e795700a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ac8ae90 2021-05-28T00:41:49.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50d53c2c-1c62-4a0b-9ff1-e583e795700a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:49.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50d53c2c-1c62-4a0b-9ff1-e583e795700a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:41:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:41:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:41:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:41:52.110Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:41:52.665Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:41:54.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3edf28c-2cd4-4ec2-acb4-0a4055643064 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:54.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3edf28c-2cd4-4ec2-acb4-0a4055643064 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d884818 2021-05-28T00:41:54.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3edf28c-2cd4-4ec2-acb4-0a4055643064 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:54.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3edf28c-2cd4-4ec2-acb4-0a4055643064 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:41:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:41:57.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41dbdba7-c64b-497c-bf8e-8ed4ab8305b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:57.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41dbdba7-c64b-497c-bf8e-8ed4ab8305b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3233a568 2021-05-28T00:41:57.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41dbdba7-c64b-497c-bf8e-8ed4ab8305b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:57.838Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41dbdba7-c64b-497c-bf8e-8ed4ab8305b3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:58.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2194b42-e987-46a5-ad75-eaaeff33a0d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:58.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2194b42-e987-46a5-ad75-eaaeff33a0d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@116e990d 2021-05-28T00:41:58.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2194b42-e987-46a5-ad75-eaaeff33a0d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:58.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2194b42-e987-46a5-ad75-eaaeff33a0d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:58.577Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:41:58.580Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f2194b42-e987-46a5-ad75-eaaeff33a0d8 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:41:58.580Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=f2194b42-e987-46a5-ad75-eaaeff33a0d8 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:41:59.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc796030-72dc-4f7f-bf98-e4ddd7418462 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:41:59.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc796030-72dc-4f7f-bf98-e4ddd7418462 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e0d199e 2021-05-28T00:41:59.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc796030-72dc-4f7f-bf98-e4ddd7418462 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:41:59.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc796030-72dc-4f7f-bf98-e4ddd7418462 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:41:59.474Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=dc796030-72dc-4f7f-bf98-e4ddd7418462 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:41:59.474Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=dc796030-72dc-4f7f-bf98-e4ddd7418462 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:42:00.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdf52216-9b33-48ec-b5cc-7d2113dfa643 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:00.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdf52216-9b33-48ec-b5cc-7d2113dfa643 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@616a50cf 2021-05-28T00:42:00.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdf52216-9b33-48ec-b5cc-7d2113dfa643 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:00.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fdf52216-9b33-48ec-b5cc-7d2113dfa643 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:01.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=845993e0-cd69-4686-9b51-01b7b9c29879 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:01.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=845993e0-cd69-4686-9b51-01b7b9c29879 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4eddb16f 2021-05-28T00:42:01.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=845993e0-cd69-4686-9b51-01b7b9c29879 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:01.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=845993e0-cd69-4686-9b51-01b7b9c29879 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:01.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ea5007e-c2f4-43cc-999f-a2c624eb2eb7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:01.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ea5007e-c2f4-43cc-999f-a2c624eb2eb7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b9eef48 2021-05-28T00:42:01.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ea5007e-c2f4-43cc-999f-a2c624eb2eb7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:01.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ea5007e-c2f4-43cc-999f-a2c624eb2eb7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:02.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f9c1e63-996d-42e3-98ec-d733f5e52185 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:02.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f9c1e63-996d-42e3-98ec-d733f5e52185 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d79d24c 2021-05-28T00:42:02.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f9c1e63-996d-42e3-98ec-d733f5e52185 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:02.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f9c1e63-996d-42e3-98ec-d733f5e52185 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6ce22cd-d60f-4397-b3f7-458ac207f590 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6ce22cd-d60f-4397-b3f7-458ac207f590 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e260704 2021-05-28T00:42:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6ce22cd-d60f-4397-b3f7-458ac207f590 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6ce22cd-d60f-4397-b3f7-458ac207f590 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:04.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ede17e0-0be1-472c-9592-ea8dffae3755 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:04.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ede17e0-0be1-472c-9592-ea8dffae3755 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dd39f66 2021-05-28T00:42:04.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ede17e0-0be1-472c-9592-ea8dffae3755 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:04.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ede17e0-0be1-472c-9592-ea8dffae3755 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:04.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65c2eb5-2020-4b3c-b671-a83a0e2f652a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:04.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65c2eb5-2020-4b3c-b671-a83a0e2f652a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@416bc73b 2021-05-28T00:42:04.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65c2eb5-2020-4b3c-b671-a83a0e2f652a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:04.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65c2eb5-2020-4b3c-b671-a83a0e2f652a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:42:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:42:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4395d6-3348-4f80-9ef0-09e5eedad1a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4395d6-3348-4f80-9ef0-09e5eedad1a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c386e8 2021-05-28T00:42:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4395d6-3348-4f80-9ef0-09e5eedad1a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da4395d6-3348-4f80-9ef0-09e5eedad1a3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2995ab70-3fb0-4cf9-9a63-fbae23c1e2a7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2995ab70-3fb0-4cf9-9a63-fbae23c1e2a7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cd3af93 2021-05-28T00:42:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2995ab70-3fb0-4cf9-9a63-fbae23c1e2a7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2995ab70-3fb0-4cf9-9a63-fbae23c1e2a7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:13.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22166a0d-6370-4340-9f56-7e58b773bab3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:13.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22166a0d-6370-4340-9f56-7e58b773bab3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@dfa39c0 2021-05-28T00:42:13.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22166a0d-6370-4340-9f56-7e58b773bab3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:13.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22166a0d-6370-4340-9f56-7e58b773bab3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:16.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98c0b17d-2632-4dc0-9869-71affe5db1f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:16.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98c0b17d-2632-4dc0-9869-71affe5db1f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1237655b 2021-05-28T00:42:16.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98c0b17d-2632-4dc0-9869-71affe5db1f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:16.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98c0b17d-2632-4dc0-9869-71affe5db1f8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:19.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96382545-d2b5-4ed8-bdc9-84af8d9da327 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:19.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96382545-d2b5-4ed8-bdc9-84af8d9da327 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c5d5533 2021-05-28T00:42:19.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96382545-d2b5-4ed8-bdc9-84af8d9da327 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:19.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96382545-d2b5-4ed8-bdc9-84af8d9da327 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:42:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:42:22.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78776beb-5378-4b48-b9ce-2a93521f3e25 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:22.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78776beb-5378-4b48-b9ce-2a93521f3e25 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1de3dc1c 2021-05-28T00:42:22.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78776beb-5378-4b48-b9ce-2a93521f3e25 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:22.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78776beb-5378-4b48-b9ce-2a93521f3e25 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:25.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10320394-4e96-443d-9536-b1af495f482d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:25.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10320394-4e96-443d-9536-b1af495f482d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@418bc259 2021-05-28T00:42:25.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10320394-4e96-443d-9536-b1af495f482d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:25.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10320394-4e96-443d-9536-b1af495f482d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95618b1-edc9-4794-a3f5-4bc668bfc99c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95618b1-edc9-4794-a3f5-4bc668bfc99c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@683f7a8e 2021-05-28T00:42:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95618b1-edc9-4794-a3f5-4bc668bfc99c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:28.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a95618b1-edc9-4794-a3f5-4bc668bfc99c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:31.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b62d57-bb6c-4ee0-925c-9ca59ec47e4c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:31.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b62d57-bb6c-4ee0-925c-9ca59ec47e4c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ead4e77 2021-05-28T00:42:31.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b62d57-bb6c-4ee0-925c-9ca59ec47e4c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:31.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b62d57-bb6c-4ee0-925c-9ca59ec47e4c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:34.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed850e0c-4904-4498-8011-766d7dd80f35 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:34.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed850e0c-4904-4498-8011-766d7dd80f35 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d2d7082 2021-05-28T00:42:34.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed850e0c-4904-4498-8011-766d7dd80f35 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:34.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed850e0c-4904-4498-8011-766d7dd80f35 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:42:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:42:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:42:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:42:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:42:37.387Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:37.387Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@122bf274 2021-05-28T00:42:37.387Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:37.387Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:37.390Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb PartnerName=SDC.BE I/O exception (org.apache.http.NoHttpResponseException) caught when processing request to {s}->https://sdc-onboarding-be.onap:8445: The target server failed to respond 2021-05-28T00:42:37.390Z [BE-Health-Check-Task] INFO o.a.http.impl.execchain.RetryExec RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb PartnerName=SDC.BE Retrying request to {s}->https://sdc-onboarding-be.onap:8445 2021-05-28T00:42:37.393Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:37.393Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7730f30d-9ab4-4cdf-9cc0-e2c8d3d087eb ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:42:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccfbc0d-ab25-441a-9825-6e85d781068e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccfbc0d-ab25-441a-9825-6e85d781068e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65e9291e 2021-05-28T00:42:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccfbc0d-ab25-441a-9825-6e85d781068e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:40.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ccfbc0d-ab25-441a-9825-6e85d781068e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:40.769Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2ccfbc0d-ab25-441a-9825-6e85d781068e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:40.769Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2ccfbc0d-ab25-441a-9825-6e85d781068e ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:42:43.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b35c07c-249e-411e-9a07-8cd1123f0b60 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:43.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b35c07c-249e-411e-9a07-8cd1123f0b60 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e531613 2021-05-28T00:42:43.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b35c07c-249e-411e-9a07-8cd1123f0b60 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:43.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b35c07c-249e-411e-9a07-8cd1123f0b60 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:43.976Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8b35c07c-249e-411e-9a07-8cd1123f0b60 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:43.976Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8b35c07c-249e-411e-9a07-8cd1123f0b60 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:42:46.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6dcb5e5-6970-44f1-963a-7d1e927c7cc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:46.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6dcb5e5-6970-44f1-963a-7d1e927c7cc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b990ae0 2021-05-28T00:42:46.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6dcb5e5-6970-44f1-963a-7d1e927c7cc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:46.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6dcb5e5-6970-44f1-963a-7d1e927c7cc1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:46.383Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c6dcb5e5-6970-44f1-963a-7d1e927c7cc1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:46.384Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c6dcb5e5-6970-44f1-963a-7d1e927c7cc1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:42:49.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89064cff-2173-4552-9c24-6b2a40e592b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:49.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89064cff-2173-4552-9c24-6b2a40e592b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19dcd146 2021-05-28T00:42:49.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89064cff-2173-4552-9c24-6b2a40e592b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:49.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89064cff-2173-4552-9c24-6b2a40e592b3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:49.574Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=89064cff-2173-4552-9c24-6b2a40e592b3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:49.574Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=89064cff-2173-4552-9c24-6b2a40e592b3 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:42:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:42:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:42:52.119Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:42:52.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c15006f2-9374-4c7e-85c7-353a4825e9d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:52.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c15006f2-9374-4c7e-85c7-353a4825e9d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e87893d 2021-05-28T00:42:52.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c15006f2-9374-4c7e-85c7-353a4825e9d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:52.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c15006f2-9374-4c7e-85c7-353a4825e9d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:52.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c15006f2-9374-4c7e-85c7-353a4825e9d8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:52.866Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c15006f2-9374-4c7e-85c7-353a4825e9d8 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:42:55.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a6c0e01-8793-43da-a4dc-e4eed68c0f4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:55.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a6c0e01-8793-43da-a4dc-e4eed68c0f4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@236c7b61 2021-05-28T00:42:55.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a6c0e01-8793-43da-a4dc-e4eed68c0f4d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:55.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a6c0e01-8793-43da-a4dc-e4eed68c0f4d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:55.760Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8a6c0e01-8793-43da-a4dc-e4eed68c0f4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:55.760Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8a6c0e01-8793-43da-a4dc-e4eed68c0f4d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:42:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea21cdb-e78a-42cc-8cf8-e610d345e712 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:42:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea21cdb-e78a-42cc-8cf8-e610d345e712 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e86ba2d 2021-05-28T00:42:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea21cdb-e78a-42cc-8cf8-e610d345e712 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:42:58.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea21cdb-e78a-42cc-8cf8-e610d345e712 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:42:58.773Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4ea21cdb-e78a-42cc-8cf8-e610d345e712 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) 2021-05-28T00:42:58.773Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4ea21cdb-e78a-42cc-8cf8-e610d345e712 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.HttpHostConnectException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Connection refused (Connection refused) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:159) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.base/java.net.PlainSocketImpl.socketConnect(Native Method) at java.base/java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) at java.base/java.net.AbstractPlainSocketImpl.connect(Unknown Source) at java.base/java.net.SocksSocketImpl.connect(Unknown Source) at java.base/java.net.Socket.connect(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:339) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=637f96f6-cae5-4d29-b670-4ee0089183b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=637f96f6-cae5-4d29-b670-4ee0089183b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c7cb807 2021-05-28T00:43:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=637f96f6-cae5-4d29-b670-4ee0089183b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=637f96f6-cae5-4d29-b670-4ee0089183b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:04.779Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=637f96f6-cae5-4d29-b670-4ee0089183b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:04.779Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=637f96f6-cae5-4d29-b670-4ee0089183b0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:43:05.518Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:43:05.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450e4da-1831-4303-9bd9-25911b40271f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:05.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450e4da-1831-4303-9bd9-25911b40271f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b319fb0 2021-05-28T00:43:05.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450e4da-1831-4303-9bd9-25911b40271f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:05.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450e4da-1831-4303-9bd9-25911b40271f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:08.666Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=d450e4da-1831-4303-9bd9-25911b40271f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:08.667Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=d450e4da-1831-4303-9bd9-25911b40271f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:09.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3507fda7-0ad9-4735-8459-cefda5069fed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:09.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3507fda7-0ad9-4735-8459-cefda5069fed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35149388 2021-05-28T00:43:09.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3507fda7-0ad9-4735-8459-cefda5069fed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:09.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3507fda7-0ad9-4735-8459-cefda5069fed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:12.289Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3507fda7-0ad9-4735-8459-cefda5069fed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:12.289Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3507fda7-0ad9-4735-8459-cefda5069fed ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:13.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c990a95-c85b-44bc-89cf-871790fcc4d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:13.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c990a95-c85b-44bc-89cf-871790fcc4d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a88f835 2021-05-28T00:43:13.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c990a95-c85b-44bc-89cf-871790fcc4d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:13.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c990a95-c85b-44bc-89cf-871790fcc4d9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:16.075Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=9c990a95-c85b-44bc-89cf-871790fcc4d9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:16.075Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=9c990a95-c85b-44bc-89cf-871790fcc4d9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:17.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aba5598b-d7c6-4fdd-9aa7-42d8fb5b6bb2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:17.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aba5598b-d7c6-4fdd-9aa7-42d8fb5b6bb2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a65109b 2021-05-28T00:43:17.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aba5598b-d7c6-4fdd-9aa7-42d8fb5b6bb2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:17.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aba5598b-d7c6-4fdd-9aa7-42d8fb5b6bb2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:20.075Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=aba5598b-d7c6-4fdd-9aa7-42d8fb5b6bb2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:20.076Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=aba5598b-d7c6-4fdd-9aa7-42d8fb5b6bb2 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:43:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:43:20.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf402780-c948-4210-b852-b07386a4f450 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:20.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf402780-c948-4210-b852-b07386a4f450 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@337f33c 2021-05-28T00:43:20.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf402780-c948-4210-b852-b07386a4f450 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:20.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf402780-c948-4210-b852-b07386a4f450 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:23.889Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=cf402780-c948-4210-b852-b07386a4f450 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:23.889Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=cf402780-c948-4210-b852-b07386a4f450 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ee067a6-cd72-4cdd-9754-326e3919516c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ee067a6-cd72-4cdd-9754-326e3919516c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58245cda 2021-05-28T00:43:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ee067a6-cd72-4cdd-9754-326e3919516c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ee067a6-cd72-4cdd-9754-326e3919516c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:27.778Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6ee067a6-cd72-4cdd-9754-326e3919516c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:27.778Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6ee067a6-cd72-4cdd-9754-326e3919516c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:28.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48192b6-edd3-4ebb-8578-5eff93756f81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:28.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48192b6-edd3-4ebb-8578-5eff93756f81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2558cfcc 2021-05-28T00:43:28.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48192b6-edd3-4ebb-8578-5eff93756f81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:28.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b48192b6-edd3-4ebb-8578-5eff93756f81 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:31.581Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b48192b6-edd3-4ebb-8578-5eff93756f81 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:31.581Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b48192b6-edd3-4ebb-8578-5eff93756f81 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:32.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75779ce8-6e35-49b4-8611-ccc7ea70bae5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:32.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75779ce8-6e35-49b4-8611-ccc7ea70bae5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e80eaa1 2021-05-28T00:43:32.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75779ce8-6e35-49b4-8611-ccc7ea70bae5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:32.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75779ce8-6e35-49b4-8611-ccc7ea70bae5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:35.466Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=75779ce8-6e35-49b4-8611-ccc7ea70bae5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out 2021-05-28T00:43:35.467Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=75779ce8-6e35-49b4-8611-ccc7ea70bae5 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to sdc-onboarding-be.onap:8445 [sdc-onboarding-be.onap/10.96.2.153] failed: Read timed out at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151) at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:359) at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:381) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source) at java.base/sun.security.ssl.SSLTransport.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.decode(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(Unknown Source) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:396) at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:355) at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:142) ... 22 common frames omitted 2021-05-28T00:43:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:43:35.524Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:43:36.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=235f51c7-c6d2-4040-98ec-d44707d613a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:36.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=235f51c7-c6d2-4040-98ec-d44707d613a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27cb0fe8 2021-05-28T00:43:36.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=235f51c7-c6d2-4040-98ec-d44707d613a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:36.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=235f51c7-c6d2-4040-98ec-d44707d613a4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:43:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:43:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:43:39.559Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=235f51c7-c6d2-4040-98ec-d44707d613a4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:43:39.560Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=235f51c7-c6d2-4040-98ec-d44707d613a4 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:43:41.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a409f14a-3255-4997-9f08-819f418fc69a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:41.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a409f14a-3255-4997-9f08-819f418fc69a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f4def5a 2021-05-28T00:43:41.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a409f14a-3255-4997-9f08-819f418fc69a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:41.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a409f14a-3255-4997-9f08-819f418fc69a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:44.108Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a409f14a-3255-4997-9f08-819f418fc69a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:43:44.109Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a409f14a-3255-4997-9f08-819f418fc69a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:43:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:43:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:43:45.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da835dc4-7f5d-4716-b0ca-c9db6b305f11 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:45.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da835dc4-7f5d-4716-b0ca-c9db6b305f11 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50c832b0 2021-05-28T00:43:45.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da835dc4-7f5d-4716-b0ca-c9db6b305f11 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:45.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da835dc4-7f5d-4716-b0ca-c9db6b305f11 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:46.284Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=da835dc4-7f5d-4716-b0ca-c9db6b305f11 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:43:46.284Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=da835dc4-7f5d-4716-b0ca-c9db6b305f11 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:43:46.679Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:43:47.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b767fe5-ce98-4d2d-8315-b7192efcba60 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:47.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b767fe5-ce98-4d2d-8315-b7192efcba60 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38d06c97 2021-05-28T00:43:47.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b767fe5-ce98-4d2d-8315-b7192efcba60 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:47.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b767fe5-ce98-4d2d-8315-b7192efcba60 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:47.482Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=1b767fe5-ce98-4d2d-8315-b7192efcba60 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:43:47.483Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=1b767fe5-ce98-4d2d-8315-b7192efcba60 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:43:48.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3350523e-1ee9-476d-9853-534f9c7c4eb5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:48.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3350523e-1ee9-476d-9853-534f9c7c4eb5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9057b9e 2021-05-28T00:43:48.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3350523e-1ee9-476d-9853-534f9c7c4eb5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:48.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3350523e-1ee9-476d-9853-534f9c7c4eb5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:48.879Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3350523e-1ee9-476d-9853-534f9c7c4eb5 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:43:48.879Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3350523e-1ee9-476d-9853-534f9c7c4eb5 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:43:50.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d029229f-86d9-42a0-9cee-acf7b4cd5955 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:50.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d029229f-86d9-42a0-9cee-acf7b4cd5955 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18c995bc 2021-05-28T00:43:50.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d029229f-86d9-42a0-9cee-acf7b4cd5955 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:50.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d029229f-86d9-42a0-9cee-acf7b4cd5955 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:43:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:43:51.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a38807b-555a-4463-8dc4-1c78e34f477f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:51.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a38807b-555a-4463-8dc4-1c78e34f477f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b4283a8 2021-05-28T00:43:51.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a38807b-555a-4463-8dc4-1c78e34f477f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:51.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a38807b-555a-4463-8dc4-1c78e34f477f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:52.128Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:43:53.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb9dd43-0467-46c3-bb48-546385014ba3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:53.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb9dd43-0467-46c3-bb48-546385014ba3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@504dfe83 2021-05-28T00:43:53.189Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb9dd43-0467-46c3-bb48-546385014ba3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:53.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb9dd43-0467-46c3-bb48-546385014ba3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:54.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d59889ed-3de9-4d0c-84de-094397684428 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:54.976Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d59889ed-3de9-4d0c-84de-094397684428 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72b6a69f 2021-05-28T00:43:54.976Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d59889ed-3de9-4d0c-84de-094397684428 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:54.976Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d59889ed-3de9-4d0c-84de-094397684428 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:56.363Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0c03a4f-58c5-42d2-9c86-34ac8c2d47a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:56.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0c03a4f-58c5-42d2-9c86-34ac8c2d47a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2606b900 2021-05-28T00:43:56.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0c03a4f-58c5-42d2-9c86-34ac8c2d47a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:56.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0c03a4f-58c5-42d2-9c86-34ac8c2d47a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:43:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4504e283-ed89-43af-a47d-7befc9b296ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:43:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4504e283-ed89-43af-a47d-7befc9b296ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59a631ab 2021-05-28T00:43:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4504e283-ed89-43af-a47d-7befc9b296ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:43:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4504e283-ed89-43af-a47d-7befc9b296ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26ac192-0624-4475-b400-5adb29af85e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26ac192-0624-4475-b400-5adb29af85e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@662065fa 2021-05-28T00:44:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26ac192-0624-4475-b400-5adb29af85e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26ac192-0624-4475-b400-5adb29af85e2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd6eee8-6ea6-4569-b0a5-d88ba773515b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd6eee8-6ea6-4569-b0a5-d88ba773515b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2165b5ff 2021-05-28T00:44:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd6eee8-6ea6-4569-b0a5-d88ba773515b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:04.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd6eee8-6ea6-4569-b0a5-d88ba773515b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:44:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:44:07.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264969bc-9bff-455a-8d68-7c263b206ed2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:07.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264969bc-9bff-455a-8d68-7c263b206ed2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3238d66a 2021-05-28T00:44:07.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264969bc-9bff-455a-8d68-7c263b206ed2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:07.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=264969bc-9bff-455a-8d68-7c263b206ed2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a767ea31-3048-4434-8056-8bdec2f869f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a767ea31-3048-4434-8056-8bdec2f869f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a7b92ef 2021-05-28T00:44:10.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a767ea31-3048-4434-8056-8bdec2f869f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:10.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a767ea31-3048-4434-8056-8bdec2f869f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:14.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dc02132-7220-4b45-bc5d-3658be4c44f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:14.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dc02132-7220-4b45-bc5d-3658be4c44f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f1123ef 2021-05-28T00:44:14.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dc02132-7220-4b45-bc5d-3658be4c44f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:14.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dc02132-7220-4b45-bc5d-3658be4c44f4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:16.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03d2e9f3-756e-4d27-a805-46c2f1180008 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:16.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03d2e9f3-756e-4d27-a805-46c2f1180008 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3357a3bf 2021-05-28T00:44:16.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03d2e9f3-756e-4d27-a805-46c2f1180008 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:16.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03d2e9f3-756e-4d27-a805-46c2f1180008 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:19.269Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7cfbf72b-6225-4820-adde-7c8133114c5b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:19.268615Z InvocationID=7f23fd79-5070-4377-9218-6dbfad731c0d LogTimestamp=2021-05-28T00:44:19.268906Z Entering 2021-05-28T00:44:19.270Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7cfbf72b-6225-4820-adde-7c8133114c5b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:19.268615Z InvocationID=7f23fd79-5070-4377-9218-6dbfad731c0d LogTimestamp=2021-05-28T00:44:19.268906Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:44:19.378Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=107 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:44:19.270Z InvocationID=a966620f-e380-4466-871d-ef0f37f515d1 LogTimestamp=2021-05-28T00:44:19.377980Z ResponseDesc=OK Exiting. 2021-05-28T00:44:19.378Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:44:19.480Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=64656 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:43:14.824Z InvocationID=ad565e55-4aef-4dc9-b790-0ed1d24b9da5 ErrorCode=500 LogTimestamp=2021-05-28T00:44:19.480301Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:19.481Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:44:19.580Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=14b75db6-6278-4fc6-8d4e-c6581e6ef093 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:19.580475Z InvocationID=396aab4e-48cc-4396-abfc-ef009128a73d LogTimestamp=2021-05-28T00:44:19.580522Z Entering 2021-05-28T00:44:19.580Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=14b75db6-6278-4fc6-8d4e-c6581e6ef093 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:19.580475Z InvocationID=396aab4e-48cc-4396-abfc-ef009128a73d LogTimestamp=2021-05-28T00:44:19.580522Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:44:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26eec097-054a-4478-a28a-e687ca852897 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26eec097-054a-4478-a28a-e687ca852897 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4728b839 2021-05-28T00:44:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26eec097-054a-4478-a28a-e687ca852897 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26eec097-054a-4478-a28a-e687ca852897 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:19.880Z [qtp580718781-17] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:44:19.580Z InvocationID=f5d73925-0895-430d-8fcd-0a7a043ef268 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@20fc564b', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 297ms 2021-05-28T00:44:19.883Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=301 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:44:19.580Z InvocationID=f5d73925-0895-430d-8fcd-0a7a043ef268 ErrorCode=409 LogTimestamp=2021-05-28T00:44:19.881432Z ResponseDesc=Conflict Exiting. 2021-05-28T00:44:19.883Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:44:20.057Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:20.057474Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:20.057Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:20.057474Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:20.058Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:44:20.169Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=13730242-f280-4534-801d-aa958c088cd9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:20.169293Z InvocationID=8d6e467e-6ff2-445a-a516-328ba03281c3 LogTimestamp=2021-05-28T00:44:20.169346Z Entering 2021-05-28T00:44:20.169Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=13730242-f280-4534-801d-aa958c088cd9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:20.169293Z InvocationID=8d6e467e-6ff2-445a-a516-328ba03281c3 LogTimestamp=2021-05-28T00:44:20.169346Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:44:20.372Z [qtp580718781-11] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:44:20.169Z InvocationID=95b02ac7-f396-4468-97df-ef129d500f27 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@20fc564b', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 202ms 2021-05-28T00:44:20.373Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=204 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:44:20.169Z InvocationID=95b02ac7-f396-4468-97df-ef129d500f27 ErrorCode=400 LogTimestamp=2021-05-28T00:44:20.373253Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:44:20.373Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:44:20.469Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:20.469501Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:20.469Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:20.469501Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:20.469Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:44:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:44:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:44:20.570Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a4cea477-7a53-415a-a6bb-e931115f1ccf InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:20.569905Z InvocationID=7e14facd-4516-4c89-b659-d84d826cb40a LogTimestamp=2021-05-28T00:44:20.569966Z Entering 2021-05-28T00:44:20.570Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a4cea477-7a53-415a-a6bb-e931115f1ccf InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:20.569905Z InvocationID=7e14facd-4516-4c89-b659-d84d826cb40a LogTimestamp=2021-05-28T00:44:20.569966Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:44:20.580Z [qtp580718781-15] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:44:20.570Z InvocationID=2ac37f39-e247-4252-8fe9-b2bb4631816b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@20fc564b', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 9ms 2021-05-28T00:44:20.581Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=11 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:44:20.570Z InvocationID=2ac37f39-e247-4252-8fe9-b2bb4631816b ErrorCode=400 LogTimestamp=2021-05-28T00:44:20.581008Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:44:20.581Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:44:20.671Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=74022 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:43:06.648Z InvocationID=fceff89a-66a5-4d15-a8b8-44d16d8e0fb1 ErrorCode=500 LogTimestamp=2021-05-28T00:44:20.670905Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:20.671Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:44:20.773Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=003f9d92-ff49-45df-a89c-25c10041fed0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:20.773664Z InvocationID=a14e7574-9d15-4404-8f8f-10b2aae26965 LogTimestamp=2021-05-28T00:44:20.773701Z Entering 2021-05-28T00:44:20.774Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=003f9d92-ff49-45df-a89c-25c10041fed0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:20.773664Z InvocationID=a14e7574-9d15-4404-8f8f-10b2aae26965 LogTimestamp=2021-05-28T00:44:20.773701Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:44:20.882Z [qtp580718781-12] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:44:20.774Z InvocationID=e85de277-089a-44fa-829d-40fed8bb4b54 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@20fc564b', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 107ms 2021-05-28T00:44:20.883Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=109 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:44:20.774Z InvocationID=e85de277-089a-44fa-829d-40fed8bb4b54 ErrorCode=409 LogTimestamp=2021-05-28T00:44:20.883362Z ResponseDesc=Conflict Exiting. 2021-05-28T00:44:20.883Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:44:21.057Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:21.057218Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:21.057Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:21.057218Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:21.057Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:44:21.088Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=df991a33-af7d-49ad-a2e0-a7160430fff8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:21.088150Z InvocationID=b972ce6b-33dc-4f4b-811a-ba00388bd36f LogTimestamp=2021-05-28T00:44:21.088226Z Entering 2021-05-28T00:44:21.088Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=df991a33-af7d-49ad-a2e0-a7160430fff8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:21.088150Z InvocationID=b972ce6b-33dc-4f4b-811a-ba00388bd36f LogTimestamp=2021-05-28T00:44:21.088226Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:44:21.280Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:44:21.088Z InvocationID=4c327c79-0e1d-4181-9b83-61c91e88d777 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@20fc564b', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 190ms 2021-05-28T00:44:21.281Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:44:21.088Z InvocationID=4c327c79-0e1d-4181-9b83-61c91e88d777 ErrorCode=409 LogTimestamp=2021-05-28T00:44:21.280909Z ResponseDesc=Conflict Exiting. 2021-05-28T00:44:21.281Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:44:21.377Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:21.377127Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:21.377Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:21.377127Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:21.378Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:44:21.475Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cfdabbba-05c3-4679-9da2-555b9f7d8b64 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:21.475612Z InvocationID=a682bdb5-cff4-4a5c-8d31-c06aad2dc649 LogTimestamp=2021-05-28T00:44:21.475654Z Entering 2021-05-28T00:44:21.475Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cfdabbba-05c3-4679-9da2-555b9f7d8b64 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:21.475612Z InvocationID=a682bdb5-cff4-4a5c-8d31-c06aad2dc649 LogTimestamp=2021-05-28T00:44:21.475654Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:44:21.590Z [qtp580718781-98] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:44:21.476Z InvocationID=6c8988fc-22b9-417b-afd5-4f0e5da8dab1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@20fc564b', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 113ms 2021-05-28T00:44:21.591Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=114 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:44:21.476Z InvocationID=6c8988fc-22b9-417b-afd5-4f0e5da8dab1 ErrorCode=409 LogTimestamp=2021-05-28T00:44:21.590897Z ResponseDesc=Conflict Exiting. 2021-05-28T00:44:21.591Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:44:22.069Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7aff6df0-b9e5-4684-bbcd-04e4d94c96f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:22.069601Z InvocationID=29bb720b-cda8-4ef8-a344-9784a1651780 LogTimestamp=2021-05-28T00:44:22.069652Z Entering 2021-05-28T00:44:22.069Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7aff6df0-b9e5-4684-bbcd-04e4d94c96f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:22.069601Z InvocationID=29bb720b-cda8-4ef8-a344-9784a1651780 LogTimestamp=2021-05-28T00:44:22.069652Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:44:22.171Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=101 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:44:22.070Z InvocationID=32ce47bf-7c20-4fc9-aa4b-fab3c36710bf LogTimestamp=2021-05-28T00:44:22.171322Z ResponseDesc=OK Exiting. 2021-05-28T00:44:22.171Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:44:22.272Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:22.272172Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:22.272Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:22.272172Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:22.272Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:44:22.378Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=56a0ddc7-319e-4d57-9684-4d9f0c429b5e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:22.378421Z InvocationID=ce8483ce-b5d6-43d5-b2a2-efc7824d2ed1 LogTimestamp=2021-05-28T00:44:22.378467Z Entering 2021-05-28T00:44:22.378Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=56a0ddc7-319e-4d57-9684-4d9f0c429b5e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:22.378421Z InvocationID=ce8483ce-b5d6-43d5-b2a2-efc7824d2ed1 LogTimestamp=2021-05-28T00:44:22.378467Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:22.572Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:22.378Z InvocationID=6669a6b7-5864-49d3-92e4-27c1b4181541 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:44:22.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:22.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4142b6d9 2021-05-28T00:44:22.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:22.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:25.775Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:44:25.775Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:44:25.776Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:44:25.776Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=39c8af04-88ea-490f-8626-ed1e8f573fa0 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:44:27.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0f8be3-8465-4364-868f-410b5b36e6c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:27.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0f8be3-8465-4364-868f-410b5b36e6c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77fafb45 2021-05-28T00:44:27.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0f8be3-8465-4364-868f-410b5b36e6c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:27.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0f8be3-8465-4364-868f-410b5b36e6c4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:28.164Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=5786 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:22.378Z InvocationID=6669a6b7-5864-49d3-92e4-27c1b4181541 LogTimestamp=2021-05-28T00:44:28.164111Z ResponseDesc=Created Exiting. 2021-05-28T00:44:28.164Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:28.174Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:28.173951Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:28.174Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:28.173951Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:28.174Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:44:28.272Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=45dd334a-d6bb-419d-80ff-782b50688a82 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:28.272249Z InvocationID=88c1711e-51fc-43a4-90c9-4a8e0abcbb68 LogTimestamp=2021-05-28T00:44:28.272309Z Entering 2021-05-28T00:44:28.272Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=45dd334a-d6bb-419d-80ff-782b50688a82 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:28.272249Z InvocationID=88c1711e-51fc-43a4-90c9-4a8e0abcbb68 LogTimestamp=2021-05-28T00:44:28.272309Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:28.277Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0f0f8be3-8465-4364-868f-410b5b36e6c4 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:44:28.277Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0f0f8be3-8465-4364-868f-410b5b36e6c4 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:44:28.470Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:28.272Z InvocationID=711f8903-ad94-4968-9b0b-0e603f72f37d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:44:29.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1c514e6-e547-419a-926c-9aba22bedc06 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:29.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1c514e6-e547-419a-926c-9aba22bedc06 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b557220 2021-05-28T00:44:29.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1c514e6-e547-419a-926c-9aba22bedc06 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:29.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1c514e6-e547-419a-926c-9aba22bedc06 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:31.374Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3102 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:28.272Z InvocationID=711f8903-ad94-4968-9b0b-0e603f72f37d LogTimestamp=2021-05-28T00:44:31.374465Z ResponseDesc=Created Exiting. 2021-05-28T00:44:31.375Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:31.664Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:31.664885Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:31.665Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:31.664885Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:31.667Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:44:31.771Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a83ee537-097a-479b-a202-2967d51c133f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:31.771437Z InvocationID=060590a8-d2bb-4b1c-8c0e-f633a883b22a LogTimestamp=2021-05-28T00:44:31.771523Z Entering 2021-05-28T00:44:31.771Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a83ee537-097a-479b-a202-2967d51c133f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:31.771437Z InvocationID=060590a8-d2bb-4b1c-8c0e-f633a883b22a LogTimestamp=2021-05-28T00:44:31.771523Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:31.956Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:31.772Z InvocationID=fc3af20b-7a26-42df-ae8a-13d03e562229 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:44:32.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9e8886-e07a-489d-9e03-453072eed380 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:32.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9e8886-e07a-489d-9e03-453072eed380 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f75f7ed 2021-05-28T00:44:32.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9e8886-e07a-489d-9e03-453072eed380 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:32.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe9e8886-e07a-489d-9e03-453072eed380 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:44:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:44:36.577Z [qtp580718781-17] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:31.772Z InvocationID=fc3af20b-7a26-42df-ae8a-13d03e562229 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:279) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1423) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1435) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:70) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:125) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:115) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:96) at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:75) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:76) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:44:36.661Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4888 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:31.772Z InvocationID=fc3af20b-7a26-42df-ae8a-13d03e562229 LogTimestamp=2021-05-28T00:44:36.660748Z ResponseDesc=Created Exiting. 2021-05-28T00:44:36.661Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:36.763Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:36.763056Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:36.763Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:36.763056Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:36.763Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:44:36.873Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=57a7c025-b656-4772-bd62-6e7f377d08e6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:36.872869Z InvocationID=1a7e4a29-5db4-4c9d-ad54-a75f6b3b7f10 LogTimestamp=2021-05-28T00:44:36.872956Z Entering 2021-05-28T00:44:36.873Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=57a7c025-b656-4772-bd62-6e7f377d08e6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:36.872869Z InvocationID=1a7e4a29-5db4-4c9d-ad54-a75f6b3b7f10 LogTimestamp=2021-05-28T00:44:36.872956Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:36.976Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:36.873Z InvocationID=309496d3-cc5b-424a-ade2-740997678d2c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:44:37.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69a285b-470f-4316-ade0-0975361861d6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:37.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69a285b-470f-4316-ade0-0975361861d6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ff51195 2021-05-28T00:44:37.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69a285b-470f-4316-ade0-0975361861d6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:37.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69a285b-470f-4316-ade0-0975361861d6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:44:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:44:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:44:38.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd4a9cd9-5afd-45fc-9447-d295a8842ba4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:38.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd4a9cd9-5afd-45fc-9447-d295a8842ba4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e79821 2021-05-28T00:44:38.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd4a9cd9-5afd-45fc-9447-d295a8842ba4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:38.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd4a9cd9-5afd-45fc-9447-d295a8842ba4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:39.663Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2790 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:36.873Z InvocationID=309496d3-cc5b-424a-ade2-740997678d2c LogTimestamp=2021-05-28T00:44:39.663684Z ResponseDesc=Created Exiting. 2021-05-28T00:44:39.664Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:39.678Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:39.678780Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:39.679Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:39.678780Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:39.679Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:44:40.088Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f793d8b3-3e29-42b9-aa73-24e2c2a03d30 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:40.087975Z InvocationID=0c6dd8f0-b45c-4331-8fbd-495117908350 LogTimestamp=2021-05-28T00:44:40.088022Z Entering 2021-05-28T00:44:40.088Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f793d8b3-3e29-42b9-aa73-24e2c2a03d30 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:40.087975Z InvocationID=0c6dd8f0-b45c-4331-8fbd-495117908350 LogTimestamp=2021-05-28T00:44:40.088022Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:40.268Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:40.088Z InvocationID=0ed258de-0ca3-44d7-82c9-1fc91569d7dc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:44:40.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a14fe960-6cb2-4a26-84fd-4be5d47dafc3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:40.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a14fe960-6cb2-4a26-84fd-4be5d47dafc3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51f33e4f 2021-05-28T00:44:40.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a14fe960-6cb2-4a26-84fd-4be5d47dafc3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:40.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a14fe960-6cb2-4a26-84fd-4be5d47dafc3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:43.373Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3284 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:40.088Z InvocationID=0ed258de-0ca3-44d7-82c9-1fc91569d7dc LogTimestamp=2021-05-28T00:44:43.372807Z ResponseDesc=Created Exiting. 2021-05-28T00:44:43.374Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:43.470Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:43.470803Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:43.470Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:43.470803Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:43.471Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:44:43.570Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=50cbe5fe-96dd-4800-8fce-46c3e06e33da InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:43.570378Z InvocationID=b1005385-1283-4923-ab18-9702757e45bb LogTimestamp=2021-05-28T00:44:43.570450Z Entering 2021-05-28T00:44:43.570Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=50cbe5fe-96dd-4800-8fce-46c3e06e33da InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:43.570378Z InvocationID=b1005385-1283-4923-ab18-9702757e45bb LogTimestamp=2021-05-28T00:44:43.570450Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:43.672Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:43.570Z InvocationID=7e3c5c82-9c03-4296-9edf-a8dfd00eefe5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:44:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a134ac-58ee-42f9-aee7-003e4b7d96a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a134ac-58ee-42f9-aee7-003e4b7d96a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51396103 2021-05-28T00:44:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a134ac-58ee-42f9-aee7-003e4b7d96a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:43.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a134ac-58ee-42f9-aee7-003e4b7d96a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:47.171Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3601 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:43.570Z InvocationID=7e3c5c82-9c03-4296-9edf-a8dfd00eefe5 LogTimestamp=2021-05-28T00:44:47.171066Z ResponseDesc=Created Exiting. 2021-05-28T00:44:47.171Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:47.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74ee19a-2c6d-48a7-8d5d-9ab2593b7435 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:47.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74ee19a-2c6d-48a7-8d5d-9ab2593b7435 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76e7ab08 2021-05-28T00:44:47.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74ee19a-2c6d-48a7-8d5d-9ab2593b7435 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:47.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74ee19a-2c6d-48a7-8d5d-9ab2593b7435 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:47.271Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:47.271094Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:47.271Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:47.271094Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:47.271Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:44:47.285Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c0bffe2a-7912-40e4-9874-2d1dd9865139 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:47.284953Z InvocationID=e70d64dd-c973-4ac5-a9e3-55e1e566e6e5 LogTimestamp=2021-05-28T00:44:47.284994Z Entering 2021-05-28T00:44:47.285Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c0bffe2a-7912-40e4-9874-2d1dd9865139 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:47.284953Z InvocationID=e70d64dd-c973-4ac5-a9e3-55e1e566e6e5 LogTimestamp=2021-05-28T00:44:47.284994Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:47.460Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:47.285Z InvocationID=a1f2cc32-bb95-4192-8d4b-5c5449f98289 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:44:49.779Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2493 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:47.285Z InvocationID=a1f2cc32-bb95-4192-8d4b-5c5449f98289 LogTimestamp=2021-05-28T00:44:49.778960Z ResponseDesc=Created Exiting. 2021-05-28T00:44:49.779Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:49.873Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:49.873527Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:49.873Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:49.873527Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:49.874Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:44:49.885Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2f7894e8-732e-49b9-ad59-773c0182e65b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:49.884932Z InvocationID=4b38c392-d953-4c0a-b65c-9c4c373b301b LogTimestamp=2021-05-28T00:44:49.884985Z Entering 2021-05-28T00:44:49.885Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2f7894e8-732e-49b9-ad59-773c0182e65b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:49.884932Z InvocationID=4b38c392-d953-4c0a-b65c-9c4c373b301b LogTimestamp=2021-05-28T00:44:49.884985Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:49.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561a3394-3f11-405f-aeaa-10df5852d33b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:49.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561a3394-3f11-405f-aeaa-10df5852d33b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b7c0365 2021-05-28T00:44:49.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561a3394-3f11-405f-aeaa-10df5852d33b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:49.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561a3394-3f11-405f-aeaa-10df5852d33b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:50.256Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:49.885Z InvocationID=880c3239-f9fd-478b-baf0-cb146b3bd3af PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:44:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:44:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:44:52.131Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:44:52.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01261d5a-da43-42bd-bd12-0581cc357391 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:52.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01261d5a-da43-42bd-bd12-0581cc357391 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b54d03 2021-05-28T00:44:52.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01261d5a-da43-42bd-bd12-0581cc357391 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:52.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01261d5a-da43-42bd-bd12-0581cc357391 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:53.268Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3383 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:49.885Z InvocationID=880c3239-f9fd-478b-baf0-cb146b3bd3af LogTimestamp=2021-05-28T00:44:53.268461Z ResponseDesc=Created Exiting. 2021-05-28T00:44:53.268Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:54.171Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:54.171596Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:54.171Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:54.171596Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:54.172Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:44:54.570Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=32392ad9-3690-414b-8b6c-554c7d98576f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:54.570360Z InvocationID=8482c97b-1c85-490a-9a13-3da914c28fee LogTimestamp=2021-05-28T00:44:54.570415Z Entering 2021-05-28T00:44:54.570Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=32392ad9-3690-414b-8b6c-554c7d98576f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:54.570360Z InvocationID=8482c97b-1c85-490a-9a13-3da914c28fee LogTimestamp=2021-05-28T00:44:54.570415Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:54.758Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:54.570Z InvocationID=cc07f534-1427-4a63-b6aa-20746b070aed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:44:55.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=605559c9-39e8-4ae5-a496-fb8bb8f3e1ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:55.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=605559c9-39e8-4ae5-a496-fb8bb8f3e1ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@191e150f 2021-05-28T00:44:55.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=605559c9-39e8-4ae5-a496-fb8bb8f3e1ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:55.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=605559c9-39e8-4ae5-a496-fb8bb8f3e1ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:44:56.863Z [qtp580718781-11] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:54.570Z InvocationID=cc07f534-1427-4a63-b6aa-20746b070aed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Lock write succeeded but took too long: duration PT0.795069S exceeded limit PT0.5S 2021-05-28T00:44:58.870Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4299 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:54.570Z InvocationID=cc07f534-1427-4a63-b6aa-20746b070aed LogTimestamp=2021-05-28T00:44:58.869885Z ResponseDesc=Created Exiting. 2021-05-28T00:44:58.870Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:44:58.982Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:58.982793Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:44:58.982Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:44:58.982793Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:44:58.983Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:44:59.071Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=867085a5-b201-4119-b6c3-8826f8717618 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:59.071347Z InvocationID=4a8474c2-c168-4946-abeb-61f782946300 LogTimestamp=2021-05-28T00:44:59.071413Z Entering 2021-05-28T00:44:59.071Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=867085a5-b201-4119-b6c3-8826f8717618 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:44:59.071347Z InvocationID=4a8474c2-c168-4946-abeb-61f782946300 LogTimestamp=2021-05-28T00:44:59.071413Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:44:59.269Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:59.071Z InvocationID=76295d80-8801-471a-a2a2-c93bb527843f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:44:59.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61145c3-efa3-4c3a-ab07-fbe951f35659 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:44:59.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61145c3-efa3-4c3a-ab07-fbe951f35659 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@526a42cf 2021-05-28T00:44:59.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61145c3-efa3-4c3a-ab07-fbe951f35659 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:44:59.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a61145c3-efa3-4c3a-ab07-fbe951f35659 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:01.772Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2701 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:44:59.071Z InvocationID=76295d80-8801-471a-a2a2-c93bb527843f LogTimestamp=2021-05-28T00:45:01.772465Z ResponseDesc=Created Exiting. 2021-05-28T00:45:01.773Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:45:01.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3af1c53-4412-4934-8e06-d87e599ff546 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:01.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3af1c53-4412-4934-8e06-d87e599ff546 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79a614cc 2021-05-28T00:45:01.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3af1c53-4412-4934-8e06-d87e599ff546 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:01.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3af1c53-4412-4934-8e06-d87e599ff546 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:01.888Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:01.888581Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:45:01.888Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:01.888581Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:01.889Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:45:02.070Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=93bdaa2d-18d1-42ad-b988-eab766333433 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:02.069843Z InvocationID=a60a999e-728a-4519-b5bf-e0c36086bae5 LogTimestamp=2021-05-28T00:45:02.069888Z Entering 2021-05-28T00:45:02.070Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=93bdaa2d-18d1-42ad-b988-eab766333433 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:02.069843Z InvocationID=a60a999e-728a-4519-b5bf-e0c36086bae5 LogTimestamp=2021-05-28T00:45:02.069888Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:45:02.256Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:02.070Z InvocationID=42946bd4-78c9-4509-a0c2-23945d742502 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:45:04.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d7cf6a9-6340-4a61-8087-ef16bedf1990 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:04.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d7cf6a9-6340-4a61-8087-ef16bedf1990 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@505293ae 2021-05-28T00:45:04.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d7cf6a9-6340-4a61-8087-ef16bedf1990 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:04.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d7cf6a9-6340-4a61-8087-ef16bedf1990 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:45:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:45:05.657Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3586 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:02.070Z InvocationID=42946bd4-78c9-4509-a0c2-23945d742502 LogTimestamp=2021-05-28T00:45:05.656680Z ResponseDesc=Created Exiting. 2021-05-28T00:45:05.657Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:45:05.770Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:05.770772Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:45:05.770Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:05.770772Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:05.771Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:45:06.213Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5916d04c-aaf8-4776-b327-ae16be1305ce InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:06.212732Z InvocationID=e0de7f1d-10b9-4f01-835b-8885ae8ec541 LogTimestamp=2021-05-28T00:45:06.212792Z Entering 2021-05-28T00:45:06.213Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5916d04c-aaf8-4776-b327-ae16be1305ce InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:06.212732Z InvocationID=e0de7f1d-10b9-4f01-835b-8885ae8ec541 LogTimestamp=2021-05-28T00:45:06.212792Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:45:06.291Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:06.213Z InvocationID=f3e8121e-87f2-4bdb-b9a9-5d7deb7a67d7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:45:08.399Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=370ac037-1c49-4d92-ad14-e5afeb7c4946 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:08.399Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=370ac037-1c49-4d92-ad14-e5afeb7c4946 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@390d42c6 2021-05-28T00:45:08.399Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=370ac037-1c49-4d92-ad14-e5afeb7c4946 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:08.399Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=370ac037-1c49-4d92-ad14-e5afeb7c4946 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:11.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=446c6abd-d024-4409-9589-e24e5ac7317e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:11.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=446c6abd-d024-4409-9589-e24e5ac7317e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39f11312 2021-05-28T00:45:11.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=446c6abd-d024-4409-9589-e24e5ac7317e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:11.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=446c6abd-d024-4409-9589-e24e5ac7317e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:12.084Z [qtp580718781-18] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:06.213Z InvocationID=f3e8121e-87f2-4bdb-b9a9-5d7deb7a67d7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Temporary exception during backend operation [CacheMutation]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:76) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:279) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1423) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1435) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:70) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:125) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:115) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:96) at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:75) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:76) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.exceptions.WriteTimeoutException.copy(WriteTimeoutException.java:113) at com.datastax.driver.core.Responses$Error.asException(Responses.java:167) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during SIMPLE write query at consistency QUORUM (2 replica were required but only 1 acknowledged the write) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:87) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:45:12.372Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=6159 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:06.213Z InvocationID=f3e8121e-87f2-4bdb-b9a9-5d7deb7a67d7 LogTimestamp=2021-05-28T00:45:12.372274Z ResponseDesc=Created Exiting. 2021-05-28T00:45:12.372Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:45:12.558Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:12.558216Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:45:12.558Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:12.558216Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:12.558Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:45:12.588Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=24be12ef-ece5-4f4b-9996-91c22999c46c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:12.588082Z InvocationID=74e0d8b6-6ddc-47b7-9f2d-123479505c8a LogTimestamp=2021-05-28T00:45:12.588140Z Entering 2021-05-28T00:45:12.588Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=24be12ef-ece5-4f4b-9996-91c22999c46c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:12.588082Z InvocationID=74e0d8b6-6ddc-47b7-9f2d-123479505c8a LogTimestamp=2021-05-28T00:45:12.588140Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:45:12.672Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:12.588Z InvocationID=ca34c7d3-444b-4b31-b6f1-a6f65cb79e31 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:45:15.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67f9cb11-80ab-4223-8314-c5a3643ebd3e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:15.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67f9cb11-80ab-4223-8314-c5a3643ebd3e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70b86094 2021-05-28T00:45:15.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67f9cb11-80ab-4223-8314-c5a3643ebd3e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:15.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67f9cb11-80ab-4223-8314-c5a3643ebd3e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:17.770Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=5182 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:12.588Z InvocationID=ca34c7d3-444b-4b31-b6f1-a6f65cb79e31 LogTimestamp=2021-05-28T00:45:17.770550Z ResponseDesc=Created Exiting. 2021-05-28T00:45:17.771Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:45:17.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62ef3f5d-5916-4c9e-bd72-58abcb7fd03e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:17.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62ef3f5d-5916-4c9e-bd72-58abcb7fd03e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34b342ce 2021-05-28T00:45:17.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62ef3f5d-5916-4c9e-bd72-58abcb7fd03e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:17.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62ef3f5d-5916-4c9e-bd72-58abcb7fd03e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:18.279Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:18.279384Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:45:18.279Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:18.279384Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:18.280Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:45:18.371Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=157f5078-0c92-41e6-a35e-fc3ed9e13a78 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:18.370825Z InvocationID=a6217ec6-9aa4-4dcf-8822-54a4d2864e4b LogTimestamp=2021-05-28T00:45:18.370887Z Entering 2021-05-28T00:45:18.371Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=157f5078-0c92-41e6-a35e-fc3ed9e13a78 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:18.370825Z InvocationID=a6217ec6-9aa4-4dcf-8822-54a4d2864e4b LogTimestamp=2021-05-28T00:45:18.370887Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:45:18.378Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:18.371Z InvocationID=4a39b909-3dce-4478-b28c-7812b4436e3f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:45:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:45:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:45:22.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbc9df6e-fc5e-4056-84e4-2ba0b103f6ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:22.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbc9df6e-fc5e-4056-84e4-2ba0b103f6ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@677b3dec 2021-05-28T00:45:22.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbc9df6e-fc5e-4056-84e4-2ba0b103f6ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:22.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbc9df6e-fc5e-4056-84e4-2ba0b103f6ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:45:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:45:23.660Z [qtp580718781-16] INFO o.j.d.util.BackendOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:18.371Z InvocationID=4a39b909-3dce-4478-b28c-7812b4436e3f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator. (SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator. (ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:674) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:1008) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:997) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1420) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1435) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:70) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:125) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:115) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:96) at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:75) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:76) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:45:25.581Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:45:25.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d061d6ba-bc70-4122-a820-7705745be589 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:25.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d061d6ba-bc70-4122-a820-7705745be589 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56a3db92 2021-05-28T00:45:25.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d061d6ba-bc70-4122-a820-7705745be589 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:25.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d061d6ba-bc70-4122-a820-7705745be589 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:25.990Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=7619 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:18.371Z InvocationID=4a39b909-3dce-4478-b28c-7812b4436e3f LogTimestamp=2021-05-28T00:45:25.990496Z ResponseDesc=Created Exiting. 2021-05-28T00:45:25.991Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:45:26.169Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:26.169025Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:45:26.169Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:26.169025Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:26.169Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:45:26.589Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d061d6ba-bc70-4122-a820-7705745be589 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:45:26.589Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=d061d6ba-bc70-4122-a820-7705745be589 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:45:26.704Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2a2baa85-11f5-4ff9-abf7-d44cad25d758 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:26.704226Z InvocationID=aca7f6ec-7c1f-4ba8-b331-cfd8487b4908 LogTimestamp=2021-05-28T00:45:26.704292Z Entering 2021-05-28T00:45:26.704Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2a2baa85-11f5-4ff9-abf7-d44cad25d758 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:26.704226Z InvocationID=aca7f6ec-7c1f-4ba8-b331-cfd8487b4908 LogTimestamp=2021-05-28T00:45:26.704292Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:45:26.774Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:26.704Z InvocationID=3314af38-e4aa-46d3-81a4-768d3551473c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:45:27.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ccf914-ad57-4487-950f-605708794bb1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:27.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ccf914-ad57-4487-950f-605708794bb1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65acbd12 2021-05-28T00:45:27.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ccf914-ad57-4487-950f-605708794bb1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:27.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ccf914-ad57-4487-950f-605708794bb1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:27.576Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f7ccf914-ad57-4487-950f-605708794bb1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:45:27.576Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f7ccf914-ad57-4487-950f-605708794bb1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8438fb6d-3293-4165-85b9-fa7caada8849 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8438fb6d-3293-4165-85b9-fa7caada8849 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b27265b 2021-05-28T00:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8438fb6d-3293-4165-85b9-fa7caada8849 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8438fb6d-3293-4165-85b9-fa7caada8849 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:29.271Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2567 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:45:26.704Z InvocationID=3314af38-e4aa-46d3-81a4-768d3551473c LogTimestamp=2021-05-28T00:45:29.271383Z ResponseDesc=Created Exiting. 2021-05-28T00:45:29.272Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:45:29.659Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a4f0681e-6f12-4551-84c5-78525a4fd9b9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:29.659254Z InvocationID=cdbec5d0-d23b-4f96-b7ed-015f20ccca29 LogTimestamp=2021-05-28T00:45:29.659315Z Entering 2021-05-28T00:45:29.659Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a4f0681e-6f12-4551-84c5-78525a4fd9b9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:29.659254Z InvocationID=cdbec5d0-d23b-4f96-b7ed-015f20ccca29 LogTimestamp=2021-05-28T00:45:29.659315Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:45:29.673Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=13 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:45:29.659Z InvocationID=c22cdbcc-9911-449c-9297-b75e7a865d27 LogTimestamp=2021-05-28T00:45:29.672984Z ResponseDesc=OK Exiting. 2021-05-28T00:45:29.673Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:45:30.177Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6b118f8b-1352-433c-bac4-5e2bb9937a66 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:30.177445Z InvocationID=4322f316-7c8f-4920-8c1e-98108ff3c43f LogTimestamp=2021-05-28T00:45:30.177494Z Entering 2021-05-28T00:45:30.178Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6b118f8b-1352-433c-bac4-5e2bb9937a66 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:30.177445Z InvocationID=4322f316-7c8f-4920-8c1e-98108ff3c43f LogTimestamp=2021-05-28T00:45:30.177494Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:45:30.756Z [qtp580718781-98] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:45:30.178Z InvocationID=c20b103d-a27a-411d-9a8f-3a6bd1a322ea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 576222387 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n576222387 100% \n 2021-05-28T00:45:30.763Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=584 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:45:30.178Z InvocationID=c20b103d-a27a-411d-9a8f-3a6bd1a322ea LogTimestamp=2021-05-28T00:45:30.762840Z ResponseDesc=OK Exiting. 2021-05-28T00:45:30.763Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:45:30.874Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=83fdf40f-9471-418f-8625-46cedfb43065 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:30.873747Z InvocationID=2e6b5755-ef49-477d-88cb-02502f300322 LogTimestamp=2021-05-28T00:45:30.873804Z Entering 2021-05-28T00:45:30.874Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=83fdf40f-9471-418f-8625-46cedfb43065 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:30.873747Z InvocationID=2e6b5755-ef49-477d-88cb-02502f300322 LogTimestamp=2021-05-28T00:45:30.873804Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:45:30.983Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:45:30.874Z InvocationID=2d5f2d9a-4970-4eec-ad95-bc64e0fd4320 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:45:31.073Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:45:30.874Z InvocationID=2d5f2d9a-4970-4eec-ad95-bc64e0fd4320 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:45:31.074Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:45:30.874Z InvocationID=2d5f2d9a-4970-4eec-ad95-bc64e0fd4320 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:45:31.074Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=200 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:45:30.874Z InvocationID=2d5f2d9a-4970-4eec-ad95-bc64e0fd4320 ErrorCode=500 LogTimestamp=2021-05-28T00:45:31.074586Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:31.075Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:45:32.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a05dc69-a128-4703-aede-98216e7d8351 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:32.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a05dc69-a128-4703-aede-98216e7d8351 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@750de68e 2021-05-28T00:45:32.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a05dc69-a128-4703-aede-98216e7d8351 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:32.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a05dc69-a128-4703-aede-98216e7d8351 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:34.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be75399c-0ecf-480a-bc36-faedf9f05dc5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:34.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be75399c-0ecf-480a-bc36-faedf9f05dc5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dbaccf6 2021-05-28T00:45:34.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be75399c-0ecf-480a-bc36-faedf9f05dc5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:34.994Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be75399c-0ecf-480a-bc36-faedf9f05dc5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:45:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:45:37.164Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:45:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:45:37.165Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:45:37.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20e781c5-178c-4edc-8479-dfd4506b9f8f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:37.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20e781c5-178c-4edc-8479-dfd4506b9f8f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dfea27e 2021-05-28T00:45:37.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20e781c5-178c-4edc-8479-dfd4506b9f8f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:37.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20e781c5-178c-4edc-8479-dfd4506b9f8f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a8dd104-94c2-4e81-addb-7792f8328a99 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a8dd104-94c2-4e81-addb-7792f8328a99 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f994e64 2021-05-28T00:45:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a8dd104-94c2-4e81-addb-7792f8328a99 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a8dd104-94c2-4e81-addb-7792f8328a99 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:41.174Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=13246fe4-506c-47fc-88e3-0cec78132e80 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:41.174466Z InvocationID=6bd0bd73-9c47-4705-93b0-e726549dcd85 LogTimestamp=2021-05-28T00:45:41.174566Z Entering 2021-05-28T00:45:41.175Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=13246fe4-506c-47fc-88e3-0cec78132e80 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:41.174466Z InvocationID=6bd0bd73-9c47-4705-93b0-e726549dcd85 LogTimestamp=2021-05-28T00:45:41.174566Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:45:41.268Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:45:41.175Z InvocationID=e27a277e-61a9-48df-b528-e0521247031e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:45:41.277Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:45:41.175Z InvocationID=e27a277e-61a9-48df-b528-e0521247031e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:45:41.278Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:45:41.175Z InvocationID=e27a277e-61a9-48df-b528-e0521247031e ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:45:41.279Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=103 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:45:41.175Z InvocationID=e27a277e-61a9-48df-b528-e0521247031e ErrorCode=500 LogTimestamp=2021-05-28T00:45:41.278787Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:41.279Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:45:44.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=215f0721-17c2-457b-b9b8-4559101fddd2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:44.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=215f0721-17c2-457b-b9b8-4559101fddd2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@553ae88f 2021-05-28T00:45:44.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=215f0721-17c2-457b-b9b8-4559101fddd2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:44.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=215f0721-17c2-457b-b9b8-4559101fddd2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:46.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:46.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e0c53a2 2021-05-28T00:45:46.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:46.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:49.867Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:45:49.868Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:45:49.869Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:45:49.869Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=87822df9-2cf5-4fcd-8d88-87c87a992c14 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:45:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:45:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:45:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:45:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:45:52.126Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:45:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:45:54.580Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7ade7e3e-277b-43e1-b33c-fd6f5326d405 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:54.579947Z InvocationID=284088a4-5f59-4de5-843c-6786a14fd2fe LogTimestamp=2021-05-28T00:45:54.580026Z Entering 2021-05-28T00:45:54.580Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7ade7e3e-277b-43e1-b33c-fd6f5326d405 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:45:54.579947Z InvocationID=284088a4-5f59-4de5-843c-6786a14fd2fe LogTimestamp=2021-05-28T00:45:54.580026Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:45:54.581Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:45:54.580Z InvocationID=969c4fe4-a4cf-48f9-8614-30cd0b99fe64 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:45:54.581Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:45:54.581Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.180 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:45:54.580Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=969c4fe4-a4cf-48f9-8614-30cd0b99fe64 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@2fd91da 2021-05-28T00:45:54.582Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:45:54.581Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.180 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:45:54.580Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=969c4fe4-a4cf-48f9-8614-30cd0b99fe64 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@2fd91da 2021-05-28T00:45:54.582Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:54.582273Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:45:54.582Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:45:54.582273Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:45:54.582Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:45:56.160Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:45:56.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a19a0aa-df8b-4bdc-8b1c-1c258d9a5b1d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:56.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a19a0aa-df8b-4bdc-8b1c-1c258d9a5b1d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7629eb4f 2021-05-28T00:45:56.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a19a0aa-df8b-4bdc-8b1c-1c258d9a5b1d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:56.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a19a0aa-df8b-4bdc-8b1c-1c258d9a5b1d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:45:57.705Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4a19a0aa-df8b-4bdc-8b1c-1c258d9a5b1d ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:45:57.705Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4a19a0aa-df8b-4bdc-8b1c-1c258d9a5b1d ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:45:59.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3c0b71-f4fb-42e5-9d68-1c779b43bcfa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:45:59.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3c0b71-f4fb-42e5-9d68-1c779b43bcfa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@713a91f7 2021-05-28T00:45:59.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3c0b71-f4fb-42e5-9d68-1c779b43bcfa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:45:59.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3c0b71-f4fb-42e5-9d68-1c779b43bcfa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:00.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d3d454a-37c5-44c0-847d-8b6b52d63dfc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:00.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d3d454a-37c5-44c0-847d-8b6b52d63dfc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2df74d31 2021-05-28T00:46:00.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d3d454a-37c5-44c0-847d-8b6b52d63dfc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:00.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d3d454a-37c5-44c0-847d-8b6b52d63dfc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:01.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6e089d-a148-45c1-9b3e-29d4f5d783c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:01.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6e089d-a148-45c1-9b3e-29d4f5d783c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@143da853 2021-05-28T00:46:01.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6e089d-a148-45c1-9b3e-29d4f5d783c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:01.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6e089d-a148-45c1-9b3e-29d4f5d783c2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:03.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacd6730-5e6f-417d-abdc-c544f35c41a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:03.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacd6730-5e6f-417d-abdc-c544f35c41a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54b70125 2021-05-28T00:46:03.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacd6730-5e6f-417d-abdc-c544f35c41a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:03.190Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacd6730-5e6f-417d-abdc-c544f35c41a0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:04.972Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d68fdc9f-aaeb-4661-a25d-f8e61c817552 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:04.971676Z InvocationID=a52dbc21-8615-4ea2-a7bf-223eaf9b24da LogTimestamp=2021-05-28T00:46:04.971776Z Entering 2021-05-28T00:46:04.972Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d68fdc9f-aaeb-4661-a25d-f8e61c817552 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:04.971676Z InvocationID=a52dbc21-8615-4ea2-a7bf-223eaf9b24da LogTimestamp=2021-05-28T00:46:04.971776Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:46:05.065Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:46:04.972Z InvocationID=e32ccffc-4c59-4be7-b70c-09a8dc519788 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:46:05.070Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:04.972Z InvocationID=e32ccffc-4c59-4be7-b70c-09a8dc519788 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:46:05.070Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:04.972Z InvocationID=e32ccffc-4c59-4be7-b70c-09a8dc519788 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:46:05.071Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=99 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:46:04.972Z InvocationID=e32ccffc-4c59-4be7-b70c-09a8dc519788 ErrorCode=500 LogTimestamp=2021-05-28T00:46:05.071291Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:46:05.071Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:46:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:46:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:46:05.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6adface-58bc-4334-a8a1-ba99ea4dcd15 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:05.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6adface-58bc-4334-a8a1-ba99ea4dcd15 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71a7876d 2021-05-28T00:46:05.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6adface-58bc-4334-a8a1-ba99ea4dcd15 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:05.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6adface-58bc-4334-a8a1-ba99ea4dcd15 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:08.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3e51fa-5c19-4b3b-8c17-e7f0487bb654 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:08.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3e51fa-5c19-4b3b-8c17-e7f0487bb654 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66aed2ad 2021-05-28T00:46:08.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3e51fa-5c19-4b3b-8c17-e7f0487bb654 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:08.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3e51fa-5c19-4b3b-8c17-e7f0487bb654 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:10.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e42157e1-00f1-47cb-90af-2127aa1627bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:10.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e42157e1-00f1-47cb-90af-2127aa1627bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1fa8811b 2021-05-28T00:46:10.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e42157e1-00f1-47cb-90af-2127aa1627bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:10.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e42157e1-00f1-47cb-90af-2127aa1627bc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:14.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24204925-fdf3-4e14-85f7-5cc99814ee92 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:14.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24204925-fdf3-4e14-85f7-5cc99814ee92 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@737a05e2 2021-05-28T00:46:14.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24204925-fdf3-4e14-85f7-5cc99814ee92 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:14.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24204925-fdf3-4e14-85f7-5cc99814ee92 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:15.179Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8600d2ec-9a64-4e59-a57c-e179aa0845f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:15.179326Z InvocationID=5eddd7fa-ea49-4278-b8b9-22f5b477f0e5 LogTimestamp=2021-05-28T00:46:15.179469Z Entering 2021-05-28T00:46:15.180Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8600d2ec-9a64-4e59-a57c-e179aa0845f9 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:15.179326Z InvocationID=5eddd7fa-ea49-4278-b8b9-22f5b477f0e5 LogTimestamp=2021-05-28T00:46:15.179469Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:46:15.277Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:46:15.180Z InvocationID=e0c6d680-3973-48a5-9f65-73f77b8b4972 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:46:15.385Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:15.180Z InvocationID=e0c6d680-3973-48a5-9f65-73f77b8b4972 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:46:15.385Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:15.180Z InvocationID=e0c6d680-3973-48a5-9f65-73f77b8b4972 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:46:15.386Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=206 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:46:15.180Z InvocationID=e0c6d680-3973-48a5-9f65-73f77b8b4972 ErrorCode=500 LogTimestamp=2021-05-28T00:46:15.386082Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:46:15.386Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:46:17.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:46:17.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:46:18.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:18.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3eab680f 2021-05-28T00:46:18.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:18.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:20.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:46:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:46:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:46:21.275Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:46:21.276Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:46:21.276Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:21.276Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e09d29a2-cfbe-45d0-b30d-3635154718f7 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:46:25.563Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ab4d7f33-e8cf-428e-806e-2db4a4cda892 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:25.563069Z InvocationID=2d684958-667b-49d8-8311-2258b930b26b LogTimestamp=2021-05-28T00:46:25.563430Z Entering 2021-05-28T00:46:25.563Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ab4d7f33-e8cf-428e-806e-2db4a4cda892 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:25.563069Z InvocationID=2d684958-667b-49d8-8311-2258b930b26b LogTimestamp=2021-05-28T00:46:25.563430Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:46:25.564Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:25.564Z InvocationID=32d3b712-fa04-4a8b-9b5b-b56e1f2853a1 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:46:25.565Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:46:25.565Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.180 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:46:25.564Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=32d3b712-fa04-4a8b-9b5b-b56e1f2853a1 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@a3c1390 2021-05-28T00:46:25.565Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:46:25.565Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.180 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:46:25.564Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=32d3b712-fa04-4a8b-9b5b-b56e1f2853a1 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@a3c1390 2021-05-28T00:46:25.565Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:46:25.565604Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:46:25.565Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:46:25.565604Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:46:25.566Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:46:25.758Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:46:26.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:26.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e00fecd 2021-05-28T00:46:26.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:26.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:29.107Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:46:29.107Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:46:29.107Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:29.107Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=2aee6a6e-43ed-4555-ae38-e140e03f1b4c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:46:30.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb8b42-fc72-4c6a-ae18-9024305fb104 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:30.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb8b42-fc72-4c6a-ae18-9024305fb104 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@560d5c17 2021-05-28T00:46:30.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb8b42-fc72-4c6a-ae18-9024305fb104 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:30.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb8b42-fc72-4c6a-ae18-9024305fb104 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:31.380Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f4bb8b42-fc72-4c6a-ae18-9024305fb104 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:31.380Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f4bb8b42-fc72-4c6a-ae18-9024305fb104 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:32.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=200a2e3d-0a73-45c2-b03b-16cb5df633ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:32.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=200a2e3d-0a73-45c2-b03b-16cb5df633ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1adcdbf2 2021-05-28T00:46:32.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=200a2e3d-0a73-45c2-b03b-16cb5df633ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:32.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=200a2e3d-0a73-45c2-b03b-16cb5df633ec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:34.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56f8916a-0887-4ca7-bdd7-2f09e6771568 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:34.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56f8916a-0887-4ca7-bdd7-2f09e6771568 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3798cb9c 2021-05-28T00:46:34.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56f8916a-0887-4ca7-bdd7-2f09e6771568 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:34.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56f8916a-0887-4ca7-bdd7-2f09e6771568 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:46:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:46:35.672Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e5266169-2865-48f4-865f-865d0e67d5ae InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:35.671671Z InvocationID=fb994c8f-0634-4b04-a145-b6bd3f4d53a8 LogTimestamp=2021-05-28T00:46:35.671764Z Entering 2021-05-28T00:46:35.672Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e5266169-2865-48f4-865f-865d0e67d5ae InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:35.671671Z InvocationID=fb994c8f-0634-4b04-a145-b6bd3f4d53a8 LogTimestamp=2021-05-28T00:46:35.671764Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:46:35.776Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:46:35.672Z InvocationID=0a33c219-9013-43fa-bb0b-523f8b23d8a4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:46:35.868Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:35.672Z InvocationID=0a33c219-9013-43fa-bb0b-523f8b23d8a4 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:46:35.868Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:35.672Z InvocationID=0a33c219-9013-43fa-bb0b-523f8b23d8a4 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:46:35.869Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=196 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:46:35.672Z InvocationID=0a33c219-9013-43fa-bb0b-523f8b23d8a4 ErrorCode=500 LogTimestamp=2021-05-28T00:46:35.868886Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:46:35.870Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:46:35.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df1f657-bdac-4808-ac65-f74b48374a7a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:35.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df1f657-bdac-4808-ac65-f74b48374a7a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bc9c49c 2021-05-28T00:46:35.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df1f657-bdac-4808-ac65-f74b48374a7a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:35.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df1f657-bdac-4808-ac65-f74b48374a7a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:46:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:46:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:46:38.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205f3077-0860-42e3-b036-70b5a2e5be43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:38.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205f3077-0860-42e3-b036-70b5a2e5be43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@250acdd9 2021-05-28T00:46:38.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205f3077-0860-42e3-b036-70b5a2e5be43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:38.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=205f3077-0860-42e3-b036-70b5a2e5be43 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:42.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca29c51-2a99-4fef-9d93-79f316ee6691 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:42.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca29c51-2a99-4fef-9d93-79f316ee6691 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59f591c1 2021-05-28T00:46:42.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca29c51-2a99-4fef-9d93-79f316ee6691 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:42.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ca29c51-2a99-4fef-9d93-79f316ee6691 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:45.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e7c7dd-c895-4fb7-8eb7-aec293050b10 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:45.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e7c7dd-c895-4fb7-8eb7-aec293050b10 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28c07548 2021-05-28T00:46:45.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e7c7dd-c895-4fb7-8eb7-aec293050b10 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:45.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e7c7dd-c895-4fb7-8eb7-aec293050b10 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:46.072Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9afdb868-188a-4bed-a9ae-377b22bdcfc6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:46.072776Z InvocationID=57c563fc-393a-4fc9-be01-0d4bc87094af LogTimestamp=2021-05-28T00:46:46.072821Z Entering 2021-05-28T00:46:46.073Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9afdb868-188a-4bed-a9ae-377b22bdcfc6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:46.072776Z InvocationID=57c563fc-393a-4fc9-be01-0d4bc87094af LogTimestamp=2021-05-28T00:46:46.072821Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:46:46.163Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:46:46.073Z InvocationID=913945a7-e0fa-46ab-baf9-119c6c6a9761 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:46:46.171Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:46.073Z InvocationID=913945a7-e0fa-46ab-baf9-119c6c6a9761 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:46:46.171Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:46.073Z InvocationID=913945a7-e0fa-46ab-baf9-119c6c6a9761 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:46:46.172Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:46:46.073Z InvocationID=913945a7-e0fa-46ab-baf9-119c6c6a9761 ErrorCode=500 LogTimestamp=2021-05-28T00:46:46.171794Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:46:46.172Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:46:47.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98f7412-1269-442b-afb4-eee0c019f00a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:47.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98f7412-1269-442b-afb4-eee0c019f00a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6de8a9e8 2021-05-28T00:46:47.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98f7412-1269-442b-afb4-eee0c019f00a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:47.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98f7412-1269-442b-afb4-eee0c019f00a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:48.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:48.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e7fc589 2021-05-28T00:46:48.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:48.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:46:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:46:51.974Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:46:51.975Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:46:51.975Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:51.976Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=81a640a3-3eea-4bcc-b058-fa612f26dab0 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:52.125Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:46:56.267Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e8f88e21-1f50-4ef8-9554-0d9ca6b79fda InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:56.267457Z InvocationID=682612d2-0540-473f-b085-2ff385fdf530 LogTimestamp=2021-05-28T00:46:56.267505Z Entering 2021-05-28T00:46:56.268Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e8f88e21-1f50-4ef8-9554-0d9ca6b79fda InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:46:56.267457Z InvocationID=682612d2-0540-473f-b085-2ff385fdf530 LogTimestamp=2021-05-28T00:46:56.267505Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:46:56.359Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:46:56.268Z InvocationID=e6741448-58d8-4dd6-a967-16cf3311d8e3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:46:56.364Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:56.268Z InvocationID=e6741448-58d8-4dd6-a967-16cf3311d8e3 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:46:56.365Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:46:56.268Z InvocationID=e6741448-58d8-4dd6-a967-16cf3311d8e3 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:46:56.365Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=97 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:46:56.268Z InvocationID=e6741448-58d8-4dd6-a967-16cf3311d8e3 ErrorCode=500 LogTimestamp=2021-05-28T00:46:56.365699Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:46:56.366Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:46:57.031Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b75c6542-b29f-4dac-bd25-d288071c7365 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:57.031Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b75c6542-b29f-4dac-bd25-d288071c7365 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bf1341e 2021-05-28T00:46:57.031Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b75c6542-b29f-4dac-bd25-d288071c7365 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:57.031Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b75c6542-b29f-4dac-bd25-d288071c7365 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:46:57.294Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b75c6542-b29f-4dac-bd25-d288071c7365 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:57.294Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b75c6542-b29f-4dac-bd25-d288071c7365 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:46:58.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e76b80-e5c5-4ec9-8d47-d01026b837bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:46:58.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e76b80-e5c5-4ec9-8d47-d01026b837bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68bf301a 2021-05-28T00:46:58.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e76b80-e5c5-4ec9-8d47-d01026b837bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:46:58.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e76b80-e5c5-4ec9-8d47-d01026b837bb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:00.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c67dbe4-d66b-46a5-8013-90240e02fdab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:00.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c67dbe4-d66b-46a5-8013-90240e02fdab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@88cf309 2021-05-28T00:47:00.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c67dbe4-d66b-46a5-8013-90240e02fdab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:00.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c67dbe4-d66b-46a5-8013-90240e02fdab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:03.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7280a558-c26a-45d7-b38a-0125ddbfe0d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:03.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7280a558-c26a-45d7-b38a-0125ddbfe0d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a3f77c2 2021-05-28T00:47:03.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7280a558-c26a-45d7-b38a-0125ddbfe0d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:03.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7280a558-c26a-45d7-b38a-0125ddbfe0d3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:04.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3e02873-815d-4093-8828-98fe052e4748 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:04.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3e02873-815d-4093-8828-98fe052e4748 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@297cf416 2021-05-28T00:47:04.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3e02873-815d-4093-8828-98fe052e4748 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:04.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3e02873-815d-4093-8828-98fe052e4748 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:47:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:47:06.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d1d5108-a84e-4328-997b-7a8fe19aaf65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:06.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d1d5108-a84e-4328-997b-7a8fe19aaf65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38522ec3 2021-05-28T00:47:06.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d1d5108-a84e-4328-997b-7a8fe19aaf65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:06.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d1d5108-a84e-4328-997b-7a8fe19aaf65 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:06.867Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fb6b0b7e-ec23-4150-b965-3e6354e4c6bb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:47:06.867446Z InvocationID=2b6c1cc7-9cd4-4169-b2b0-1f96a56172e8 LogTimestamp=2021-05-28T00:47:06.867498Z Entering 2021-05-28T00:47:06.867Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fb6b0b7e-ec23-4150-b965-3e6354e4c6bb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:47:06.867446Z InvocationID=2b6c1cc7-9cd4-4169-b2b0-1f96a56172e8 LogTimestamp=2021-05-28T00:47:06.867498Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:47:06.954Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:47:06.868Z InvocationID=416fbe41-71ee-4fc4-a02d-192a5974bac1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:47:06.983Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:47:06.868Z InvocationID=416fbe41-71ee-4fc4-a02d-192a5974bac1 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:47:06.983Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:47:06.868Z InvocationID=416fbe41-71ee-4fc4-a02d-192a5974bac1 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:47:06.984Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=116 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:47:06.868Z InvocationID=416fbe41-71ee-4fc4-a02d-192a5974bac1 ErrorCode=500 LogTimestamp=2021-05-28T00:47:06.984332Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:47:06.984Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:47:09.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29b022b3-74bd-465b-a282-16979f68b098 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:09.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29b022b3-74bd-465b-a282-16979f68b098 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57a9867b 2021-05-28T00:47:09.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29b022b3-74bd-465b-a282-16979f68b098 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:09.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29b022b3-74bd-465b-a282-16979f68b098 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:47:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:47:13.786Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:47:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be41ec00-ddae-4154-bea9-ab15ab7abe65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be41ec00-ddae-4154-bea9-ab15ab7abe65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46b97a2b 2021-05-28T00:47:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be41ec00-ddae-4154-bea9-ab15ab7abe65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be41ec00-ddae-4154-bea9-ab15ab7abe65 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:15.899Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=be41ec00-ddae-4154-bea9-ab15ab7abe65 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:15.899Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=be41ec00-ddae-4154-bea9-ab15ab7abe65 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:47:16.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:16.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8abeaae 2021-05-28T00:47:16.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:16.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:17.769Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=54e0a833-7d01-4c08-a365-cc00e080e250 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:47:17.768853Z InvocationID=9988214a-13e3-4515-911b-787bd2967ddf LogTimestamp=2021-05-28T00:47:17.768943Z Entering 2021-05-28T00:47:17.769Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=54e0a833-7d01-4c08-a365-cc00e080e250 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.180 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.180 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:47:17.768853Z InvocationID=9988214a-13e3-4515-911b-787bd2967ddf LogTimestamp=2021-05-28T00:47:17.768943Z localAddr=10.242.102.221 remoteAddr=10.242.170.180 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:47:17.857Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:47:17.770Z InvocationID=4d7390b1-face-40d4-90f7-d77a610c0029 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:47:17.872Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:47:17.770Z InvocationID=4d7390b1-face-40d4-90f7-d77a610c0029 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:47:17.872Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:47:17.770Z InvocationID=4d7390b1-face-40d4-90f7-d77a610c0029 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:47:17.873Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=103 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:47:17.770Z InvocationID=4d7390b1-face-40d4-90f7-d77a610c0029 ErrorCode=500 LogTimestamp=2021-05-28T00:47:17.873307Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:47:17.873Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:47:19.867Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:47:19.868Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:47:19.868Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:19.868Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6b2e3798-7577-4b43-b443-1849e50a27e1 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:47:20.565Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:47:22.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344a33d5-6618-4d7f-b62d-f525a5abbca1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:22.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344a33d5-6618-4d7f-b62d-f525a5abbca1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2126e94e 2021-05-28T00:47:22.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344a33d5-6618-4d7f-b62d-f525a5abbca1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:22.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344a33d5-6618-4d7f-b62d-f525a5abbca1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:47:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:47:25.494Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=344a33d5-6618-4d7f-b62d-f525a5abbca1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:47:25.495Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=344a33d5-6618-4d7f-b62d-f525a5abbca1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:47:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:47:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:47:30.561Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:47:30.570Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:47:32.562Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:47:33.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53706721-5f0a-4af9-a93c-730f1431add4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:33.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53706721-5f0a-4af9-a93c-730f1431add4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37c27402 2021-05-28T00:47:33.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53706721-5f0a-4af9-a93c-730f1431add4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:33.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53706721-5f0a-4af9-a93c-730f1431add4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:47:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:47:35.777Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=53706721-5f0a-4af9-a93c-730f1431add4 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:35.777Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=53706721-5f0a-4af9-a93c-730f1431add4 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:47:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:47:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:47:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:47:37.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cffda71b-5192-42c1-88a2-7fff03f23a21 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:37.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cffda71b-5192-42c1-88a2-7fff03f23a21 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7691ff04 2021-05-28T00:47:37.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cffda71b-5192-42c1-88a2-7fff03f23a21 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:37.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cffda71b-5192-42c1-88a2-7fff03f23a21 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:38.984Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=cffda71b-5192-42c1-88a2-7fff03f23a21 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:38.984Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=cffda71b-5192-42c1-88a2-7fff03f23a21 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:39.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b7b4dee-2507-4cbd-bb90-372379afa7f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:39.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b7b4dee-2507-4cbd-bb90-372379afa7f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@757fddc8 2021-05-28T00:47:39.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b7b4dee-2507-4cbd-bb90-372379afa7f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:39.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b7b4dee-2507-4cbd-bb90-372379afa7f8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:42.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56564c76-b9bb-48eb-a6bc-958d1da27cbd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:42.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56564c76-b9bb-48eb-a6bc-958d1da27cbd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5139d501 2021-05-28T00:47:42.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56564c76-b9bb-48eb-a6bc-958d1da27cbd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:42.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56564c76-b9bb-48eb-a6bc-958d1da27cbd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:45.401Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48b3a324-0b30-4ba9-9215-1090f7b0d5ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:45.401Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48b3a324-0b30-4ba9-9215-1090f7b0d5ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69dce48e 2021-05-28T00:47:45.401Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48b3a324-0b30-4ba9-9215-1090f7b0d5ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:45.401Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48b3a324-0b30-4ba9-9215-1090f7b0d5ed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:47:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:47:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:47:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:47:51.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:47:51.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10cc4f6e 2021-05-28T00:47:51.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:47:51.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:47:51.947Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:47:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:47:54.481Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:47:54.481Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:47:54.482Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:54.482Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=68ba237f-0168-41ca-93f9-fe1c2d4f4f90 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:47:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:47:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:48:01.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=788b4aaa-a283-46c0-ad80-d89e046c128d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:01.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=788b4aaa-a283-46c0-ad80-d89e046c128d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1bd65122 2021-05-28T00:48:01.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=788b4aaa-a283-46c0-ad80-d89e046c128d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:01.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=788b4aaa-a283-46c0-ad80-d89e046c128d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:02.278Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:48:02.280Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=788b4aaa-a283-46c0-ad80-d89e046c128d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:02.280Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=788b4aaa-a283-46c0-ad80-d89e046c128d ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:48:03.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94b1fe4-aad2-4481-93ab-deb0f1ef9dc7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:03.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94b1fe4-aad2-4481-93ab-deb0f1ef9dc7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a2e6205 2021-05-28T00:48:03.172Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94b1fe4-aad2-4481-93ab-deb0f1ef9dc7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:03.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c94b1fe4-aad2-4481-93ab-deb0f1ef9dc7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:03.779Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c94b1fe4-aad2-4481-93ab-deb0f1ef9dc7 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:03.779Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c94b1fe4-aad2-4481-93ab-deb0f1ef9dc7 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:04.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9730d1b-ff5c-4849-a51a-98f8aae1c09c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:04.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9730d1b-ff5c-4849-a51a-98f8aae1c09c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@786bfb11 2021-05-28T00:48:04.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9730d1b-ff5c-4849-a51a-98f8aae1c09c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:04.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9730d1b-ff5c-4849-a51a-98f8aae1c09c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:48:05.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:48:06.193Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54e602dd-81f6-485d-a0a8-a60042e8567b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:06.193Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54e602dd-81f6-485d-a0a8-a60042e8567b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@519dea7f 2021-05-28T00:48:06.193Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54e602dd-81f6-485d-a0a8-a60042e8567b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:06.194Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54e602dd-81f6-485d-a0a8-a60042e8567b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:07.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e95e778b-8dd2-4a16-850a-2de2a4fdf82a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:07.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e95e778b-8dd2-4a16-850a-2de2a4fdf82a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15f5e2f6 2021-05-28T00:48:07.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e95e778b-8dd2-4a16-850a-2de2a4fdf82a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:07.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e95e778b-8dd2-4a16-850a-2de2a4fdf82a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:10.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1433fda-5dc8-4756-b4a3-1b554a82e163 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:10.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1433fda-5dc8-4756-b4a3-1b554a82e163 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75569cfa 2021-05-28T00:48:10.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1433fda-5dc8-4756-b4a3-1b554a82e163 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:10.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1433fda-5dc8-4756-b4a3-1b554a82e163 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:11.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:11.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f8e22f4 2021-05-28T00:48:11.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:11.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:14.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:48:14.471Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:48:14.766Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:48:14.766Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:48:14.766Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:14.766Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a1e2b304-b966-4de0-bbcb-aac4c721632d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:16.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=964fb186-42ae-4620-bf01-c8cbe7a3288e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:16.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=964fb186-42ae-4620-bf01-c8cbe7a3288e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@376485be 2021-05-28T00:48:16.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=964fb186-42ae-4620-bf01-c8cbe7a3288e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:16.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=964fb186-42ae-4620-bf01-c8cbe7a3288e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:17.005Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:48:17.484Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=964fb186-42ae-4620-bf01-c8cbe7a3288e ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:17.484Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=964fb186-42ae-4620-bf01-c8cbe7a3288e ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:48:18.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66bae68d-b937-4b78-8a37-d5751eaf5a08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:18.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66bae68d-b937-4b78-8a37-d5751eaf5a08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f84cf88 2021-05-28T00:48:18.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66bae68d-b937-4b78-8a37-d5751eaf5a08 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:18.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66bae68d-b937-4b78-8a37-d5751eaf5a08 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:18.576Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=66bae68d-b937-4b78-8a37-d5751eaf5a08 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:18.577Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=66bae68d-b937-4b78-8a37-d5751eaf5a08 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:19.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475686e4-f557-4887-b34d-a85c74a3ad96 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:19.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475686e4-f557-4887-b34d-a85c74a3ad96 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3123d50e 2021-05-28T00:48:19.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475686e4-f557-4887-b34d-a85c74a3ad96 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:19.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=475686e4-f557-4887-b34d-a85c74a3ad96 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:20.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7266d88f-7ad4-45a9-96d2-3b544bcfc65e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:20.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7266d88f-7ad4-45a9-96d2-3b544bcfc65e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22f9efe1 2021-05-28T00:48:20.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7266d88f-7ad4-45a9-96d2-3b544bcfc65e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:20.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7266d88f-7ad4-45a9-96d2-3b544bcfc65e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:48:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:48:21.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c30731-997d-4e47-80d9-10db30755a8d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:21.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c30731-997d-4e47-80d9-10db30755a8d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d3d7b75 2021-05-28T00:48:21.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c30731-997d-4e47-80d9-10db30755a8d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:21.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c30731-997d-4e47-80d9-10db30755a8d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:23.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ed5f714-8379-4c75-a048-28439d112bb7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:23.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ed5f714-8379-4c75-a048-28439d112bb7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64ec15d5 2021-05-28T00:48:23.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ed5f714-8379-4c75-a048-28439d112bb7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:23.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ed5f714-8379-4c75-a048-28439d112bb7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:24.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9c0ba18-1bac-45d6-abc2-ff166a647205 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:24.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9c0ba18-1bac-45d6-abc2-ff166a647205 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b4827e3 2021-05-28T00:48:24.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9c0ba18-1bac-45d6-abc2-ff166a647205 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:24.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9c0ba18-1bac-45d6-abc2-ff166a647205 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50529aa4-0aa6-4bba-ab2d-731bac214951 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50529aa4-0aa6-4bba-ab2d-731bac214951 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26a35f8c 2021-05-28T00:48:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50529aa4-0aa6-4bba-ab2d-731bac214951 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50529aa4-0aa6-4bba-ab2d-731bac214951 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:26.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ed07dff-6a51-47d9-94a3-a40b2b38d7b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:26.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ed07dff-6a51-47d9-94a3-a40b2b38d7b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@637efe 2021-05-28T00:48:26.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ed07dff-6a51-47d9-94a3-a40b2b38d7b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:26.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ed07dff-6a51-47d9-94a3-a40b2b38d7b7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:28.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:28.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a5919a 2021-05-28T00:48:28.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:28.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:31.761Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:48:31.761Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:48:31.761Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:31.762Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c85cd7b2-e2d1-4368-a847-e39905702237 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:32.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344eb575-51b1-4647-b795-06fa459c8f74 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:32.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344eb575-51b1-4647-b795-06fa459c8f74 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e58d9d2 2021-05-28T00:48:32.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344eb575-51b1-4647-b795-06fa459c8f74 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:32.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=344eb575-51b1-4647-b795-06fa459c8f74 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:33.163Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=344eb575-51b1-4647-b795-06fa459c8f74 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:33.164Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=344eb575-51b1-4647-b795-06fa459c8f74 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:48:33.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d7b1228-55ef-4648-9b54-755d0626fe9b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:33.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d7b1228-55ef-4648-9b54-755d0626fe9b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29b4b48d 2021-05-28T00:48:33.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d7b1228-55ef-4648-9b54-755d0626fe9b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:33.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d7b1228-55ef-4648-9b54-755d0626fe9b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:35.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0bdcaa-27f2-405b-9805-6f31fdf64004 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:35.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0bdcaa-27f2-405b-9805-6f31fdf64004 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b5c960 2021-05-28T00:48:35.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0bdcaa-27f2-405b-9805-6f31fdf64004 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:35.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0bdcaa-27f2-405b-9805-6f31fdf64004 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:48:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:48:37.164Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:48:37.165Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:48:37.165Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:48:38.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72b792d6-507e-41df-b53e-5b7521552c36 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:38.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72b792d6-507e-41df-b53e-5b7521552c36 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@496b06cd 2021-05-28T00:48:38.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72b792d6-507e-41df-b53e-5b7521552c36 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:38.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72b792d6-507e-41df-b53e-5b7521552c36 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:40.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6efb42a6-a2a3-4785-abb5-062626170bab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:40.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6efb42a6-a2a3-4785-abb5-062626170bab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b7e3c9 2021-05-28T00:48:40.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6efb42a6-a2a3-4785-abb5-062626170bab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:40.986Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6efb42a6-a2a3-4785-abb5-062626170bab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=150547f4-d75e-4d7f-b514-f6d87fb8bd5f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=150547f4-d75e-4d7f-b514-f6d87fb8bd5f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@147bd0dc 2021-05-28T00:48:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=150547f4-d75e-4d7f-b514-f6d87fb8bd5f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=150547f4-d75e-4d7f-b514-f6d87fb8bd5f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:46.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c3c59c6-0939-4e93-b2fe-c2eb51d99e9c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:46.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c3c59c6-0939-4e93-b2fe-c2eb51d99e9c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@623afc68 2021-05-28T00:48:46.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c3c59c6-0939-4e93-b2fe-c2eb51d99e9c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:46.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c3c59c6-0939-4e93-b2fe-c2eb51d99e9c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:48:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:48:51.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa43afa-e4e7-4665-8cd4-3591eeb400e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:51.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa43afa-e4e7-4665-8cd4-3591eeb400e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ec9aeb2 2021-05-28T00:48:51.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa43afa-e4e7-4665-8cd4-3591eeb400e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:51.001Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa43afa-e4e7-4665-8cd4-3591eeb400e1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:52.213Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:48:54.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef5fe0cf-b730-4dc8-b254-2d67413e470f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:54.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef5fe0cf-b730-4dc8-b254-2d67413e470f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44daa309 2021-05-28T00:48:54.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef5fe0cf-b730-4dc8-b254-2d67413e470f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:54.557Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef5fe0cf-b730-4dc8-b254-2d67413e470f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:48:57.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:48:57.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e6d767e 2021-05-28T00:48:57.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:48:57.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:00.181Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:49:00.181Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:49:00.181Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:00.181Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7b858188-2f63-4609-bdde-ee22dd3d648c ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:03.363Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c81ffab-feac-41a8-b6d0-e002244c8ef5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:03.363Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c81ffab-feac-41a8-b6d0-e002244c8ef5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b010267 2021-05-28T00:49:03.363Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c81ffab-feac-41a8-b6d0-e002244c8ef5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:03.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c81ffab-feac-41a8-b6d0-e002244c8ef5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:04.174Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=5c81ffab-feac-41a8-b6d0-e002244c8ef5 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:04.174Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=5c81ffab-feac-41a8-b6d0-e002244c8ef5 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:49:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:49:06.404Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=533d79dd-d4a8-4c13-b00b-951603e426f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:06.404Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=533d79dd-d4a8-4c13-b00b-951603e426f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@195daf78 2021-05-28T00:49:06.404Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=533d79dd-d4a8-4c13-b00b-951603e426f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:06.404Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=533d79dd-d4a8-4c13-b00b-951603e426f2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:08.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cafcec2-d26c-4b64-9cda-7b326170fadb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:08.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cafcec2-d26c-4b64-9cda-7b326170fadb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c66f4ab 2021-05-28T00:49:08.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cafcec2-d26c-4b64-9cda-7b326170fadb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:08.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cafcec2-d26c-4b64-9cda-7b326170fadb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39627541-95b5-4078-b0c1-86e8fc90b3f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39627541-95b5-4078-b0c1-86e8fc90b3f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f47c37a 2021-05-28T00:49:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39627541-95b5-4078-b0c1-86e8fc90b3f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:09.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39627541-95b5-4078-b0c1-86e8fc90b3f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:12.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e09a50-d3bd-4e63-932a-9623d23e81fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:12.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e09a50-d3bd-4e63-932a-9623d23e81fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3389b7c6 2021-05-28T00:49:12.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e09a50-d3bd-4e63-932a-9623d23e81fc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:12.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e09a50-d3bd-4e63-932a-9623d23e81fc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:13.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa65856-3da8-4875-89ea-9cd27e1dc913 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:13.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa65856-3da8-4875-89ea-9cd27e1dc913 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@475be8a0 2021-05-28T00:49:13.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa65856-3da8-4875-89ea-9cd27e1dc913 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:13.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7fa65856-3da8-4875-89ea-9cd27e1dc913 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=168450d7-0003-465d-a319-90ce95cfe73b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=168450d7-0003-465d-a319-90ce95cfe73b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c538f4c 2021-05-28T00:49:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=168450d7-0003-465d-a319-90ce95cfe73b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:17.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=168450d7-0003-465d-a319-90ce95cfe73b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:49:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:49:22.337Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74485e2-7f19-460f-b2be-7090f23d3093 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:22.337Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74485e2-7f19-460f-b2be-7090f23d3093 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7dfed1af 2021-05-28T00:49:22.337Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74485e2-7f19-460f-b2be-7090f23d3093 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:22.337Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74485e2-7f19-460f-b2be-7090f23d3093 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:29.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2361a082-26b7-4034-8667-51a9cff673fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:29.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2361a082-26b7-4034-8667-51a9cff673fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3376e51 2021-05-28T00:49:29.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2361a082-26b7-4034-8667-51a9cff673fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:29.486Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2361a082-26b7-4034-8667-51a9cff673fd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:49:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:49:32.491Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=2361a082-26b7-4034-8667-51a9cff673fd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:49:32.491Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=2361a082-26b7-4034-8667-51a9cff673fd ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:49:32.491Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2361a082-26b7-4034-8667-51a9cff673fd ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:32.491Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2361a082-26b7-4034-8667-51a9cff673fd ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:49:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:49:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:49:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:49:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:49:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:49:37.559Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:49:37.571Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:49:38.470Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:49:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:49:43.564Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:49:43.565Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:49:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:49:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:49:49.576Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:49:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:49:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:49:50.570Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T00:49:50.573Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 4000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:49:51.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f49662e-0482-46ed-aa6f-1b4b0b8b3029 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:51.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f49662e-0482-46ed-aa6f-1b4b0b8b3029 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1bb1887f 2021-05-28T00:49:51.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f49662e-0482-46ed-aa6f-1b4b0b8b3029 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:51.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f49662e-0482-46ed-aa6f-1b4b0b8b3029 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:49:51.863Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3f49662e-0482-46ed-aa6f-1b4b0b8b3029 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:51.863Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=3f49662e-0482-46ed-aa6f-1b4b0b8b3029 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:49:52.133Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:49:52.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38db2330-65d6-492a-9430-135c12bfa1c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:52.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38db2330-65d6-492a-9430-135c12bfa1c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7096dd 2021-05-28T00:49:52.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38db2330-65d6-492a-9430-135c12bfa1c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:52.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38db2330-65d6-492a-9430-135c12bfa1c7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T00:49:53.275Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=38db2330-65d6-492a-9430-135c12bfa1c7 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:53.275Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=38db2330-65d6-492a-9430-135c12bfa1c7 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:49:58.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78acb45b-2be9-4e7c-8daa-cfc319fd2700 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:58.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78acb45b-2be9-4e7c-8daa-cfc319fd2700 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@380d2564 2021-05-28T00:49:58.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78acb45b-2be9-4e7c-8daa-cfc319fd2700 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:58.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78acb45b-2be9-4e7c-8daa-cfc319fd2700 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:49:59.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d51be2f-ee5f-4b59-ae57-8fa90d5ab3f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:49:59.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d51be2f-ee5f-4b59-ae57-8fa90d5ab3f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f68b06a 2021-05-28T00:49:59.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d51be2f-ee5f-4b59-ae57-8fa90d5ab3f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:49:59.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d51be2f-ee5f-4b59-ae57-8fa90d5ab3f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:01.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1393c4dc-fe51-43f1-af7b-6f6fdfc41fe1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:01.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1393c4dc-fe51-43f1-af7b-6f6fdfc41fe1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66c44d30 2021-05-28T00:50:01.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1393c4dc-fe51-43f1-af7b-6f6fdfc41fe1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:01.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1393c4dc-fe51-43f1-af7b-6f6fdfc41fe1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:03.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cd81f8-531a-4d95-9411-589cfd5a5a49 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:03.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cd81f8-531a-4d95-9411-589cfd5a5a49 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6206f94b 2021-05-28T00:50:03.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cd81f8-531a-4d95-9411-589cfd5a5a49 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:03.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cd81f8-531a-4d95-9411-589cfd5a5a49 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:50:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:50:06.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=268f7e79-6e3a-4d01-bdb8-8fb1bbfcd569 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:06.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=268f7e79-6e3a-4d01-bdb8-8fb1bbfcd569 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33f1d66a 2021-05-28T00:50:06.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=268f7e79-6e3a-4d01-bdb8-8fb1bbfcd569 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:06.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=268f7e79-6e3a-4d01-bdb8-8fb1bbfcd569 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:08.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:08.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5970cb9e 2021-05-28T00:50:08.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:08.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:08.144Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01d8jedt5vwux8e9r5udvt3go684 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01wzmoogglyh4zb8ppj5lzkdl05 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0oq6v7fkr6h807w45lhah7cyv110 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011pi41u7bp6zy19zit707hbppl33 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0bqt1lt9c3xzee37dr95w7v50156 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01amkl4qfbmvqe1wc83ocltllsm128 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02fw0qlaqy19yy6wn2s8evdq3182 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01olyyysywbd6z1c5ii5g9upisl78 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0febcemcrnjfh50k9skdsao104 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0vp9c85ouocx96j7zczjmqucy58 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node017jpq799sbqub415u30yk0v0l138 2021-05-28T00:50:08.145Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0151mg3kbvxjurc7ao0q7padho48 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01eo617rdmztv4dbidrftb1s4q97 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01fek4s41ggdne12tr3jqwp0zl7115 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01hagj4osx8j7i18i757nc9xqko122 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01pp7quiugmifsfmks5k2yh2zj160 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0afo7e3dwam0w3kxlyvbb0e0e106 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01s0tcxugv8fdn1flkcdhzousjj152 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gpndhpvuo6mbziyxnedlcoos8 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0as1i0lzuhgmkpm58zj4y9qsc95 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0wx6cgerbwfn9ylaua1sv7tkl19 2021-05-28T00:50:08.146Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yvc75ardyc8b8ly24xplvhfa49 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01cpyqxznz621t1jf5cjpddanu811 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09f20rmx5xn67qm85nd53avrk116 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ak9cysb19nadbk6axscg4a3142 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node017rzmg95hvnnng3kp95b6a6mu59 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01nlaxaiswohcqznyjm8q1wp1w130 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01fumcllyt3s0wfwrnf9ersj5m101 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node016zlc65p1imqc2xa3cktonb5v143 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node04w8hnyyb64i6r3xdicw2n0rd74 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01spumtddk1qbg1lal47g4k4vr644 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node073d3v0zwyayf4nvgtypcaarv75 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02vwuy9dlezzt1ndnhgr4st4lt168 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0194v3i57aym4118k8ogm1ax1mo81 2021-05-28T00:50:08.147Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01toodso5q16v1twn1twkm0yt383 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01uvavnsm6z9hh7vo6azguvjtu65 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0faswpkjzh86x1rdiwe6kv3o5f25 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0s3k2lp9os9eb1tn7kbd6qfkip9 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01dkhnwou3rn7f1rzxk8okea9ao13 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01w1qejuth8ry97u3fj9r0ti9s12 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0d332ncg25jkw1r3fvppal3e7l56 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node041au8w00gqb08ysuvxvy7xvj155 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01np19abudeuo61vmcxta6spxvj27 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01g96lmfm2fmv51m77q7bvhdbvz114 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0t4oqqk4zgxuyj2uju3hfurbs41 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01umisqfhm9ees1u2fjge1smvcu22 2021-05-28T00:50:08.148Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01epj3gisjm0isg8q6qmqlym7l112 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0v2flgoj2t6oc1on3a139r1ozx119 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node04ehsg3dqf3lx1oqm0vcyrw0wi16 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qotw8kg89pi01dj0dlswad0dw134 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01chvayhj78kfz15a7gwcu1v3nn178 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01nf70n8jyd9g3pxikvk2vxric47 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0pabn0ujrw9vt1amtx9elhktso88 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0sfakypi2td9y1lupkcp9uzmxl18 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01swrszw5k4a9z9ik390h43u2h159 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qclcqig1ldf51rjf31f6mjevy68 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node019j7hq7dsdibxqrlf6jgikghg176 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06p1idd35wqrn15smdyjros7dd23 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mj9xxc40iztb1ivadmh6klg3b61 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06zhtsgwduegw1oc7n729k00is64 2021-05-28T00:50:08.149Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0sxp8gihrm0rmsot1zyutrpu717 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010nf0umwjd4ur9zjfs6ghef9t123 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0fy93eh1fmw2i11zpg1rglajjw35 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tgkez0ir1emvy7tvk1dro8r515 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yopgoo9qcqne9mngpp0lsn0h113 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yn7i31omf7xh1puzauxhgyifw77 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node014chkyzp5rs4z1c1ch3hkok3sc82 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node022l29gr2fls81p0nypkucj95470 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node082t0y3flki9ahgysvnczsek626 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0142x2rv8ygvnxzptbko9hckee89 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yecrmwtfwd1m6zfylbeeakia161 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010jezokyd8jpw1r9akb7cz7hyx117 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tve6fdxd2d9j1ci4a1jhf2u4h165 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ashqehc5a5gbw1mn5tsmd7es6 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01a2k1ck2qkfri1h4o3mcyxirkp129 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0odl3fxmye82gyg4kx6xre0zi158 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0x6jatxpk3flad3n4uivlfgix146 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xukpq1fwhwaynid10kjo987455 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011lcme9jwu1a210pm6bdf51spr169 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0w87t4e4uchavm69vvbuzl89r132 2021-05-28T00:50:08.150Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01h3t95ejpyavduygkkf1ulwv1172 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xej08inopxe21sc8j5kmt6lhr3 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xdg52lewyksn93jl0biz318q99 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01crq1yl1lcioh14h4tena6l733166 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ox6voq98nyd14dwn0bxf7y86180 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0uprewylej5xgr8be2fpd5yjw52 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0188dfs6it2d2z16379yzzv76it14 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01565dbz42yznx1e3jgex3op03j133 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0f2wf399y94q61c1x9qeisj61p42 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02vx8j0848mdmju0lfynzx5ep63 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01atumv8ae6azmevoi7mfmrnla109 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tbd9j96ch2uuqpm7waq1zltt90 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01cxo6m94t5zwd199jqd2bth1u8105 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ivx1r8oemiqo1r8dpr2yx617c29 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tq01jqeardsieoxe37kqh9ef174 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gzdx3j62jhddu64vij9in2k2108 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0d74xfxowze711qfo1ln90e5xt24 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qm3lckkbgol61mhyhmxl384o336 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01fsb8hf3jwzi21e0dxia3y1e7n162 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01uset85h8c9au14vq8e2cblpsq149 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010yem3bfwgryobepjdlxs3vyf32 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01fril9v7wxai91tx2k4rf2lc70154 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ebzioj8ok6jc17lc5zfu39wdf179 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0o64d8j5sj15dxcrubg4q8b6c135 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0g2q857wurvrv1vp26ua7b3mnd7 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01n89oujx5j8xmcfagthinp9j285 2021-05-28T00:50:08.151Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ly9jqggonjv9jttp8q4vo0i6177 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01r0x2hhe4vi41u71rfoe8dh0p20 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02rsua24h8ct014w56l20mg6io37 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0150imnyeyhwmz1eujbv63uoflt30 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0lq3qqzz6wu957x8fav4j5h7l53 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qm5cz7o4xdhk19qyhay6w06vr148 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01r7ds8rfdqed611uurf0e187l136 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01tuzgrnjqfeu712nxcjt7hhq0l10 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0g63awzh5ihe1jjybcbvl2z8d98 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node012vch37zyvkldlz4movtmlmnw28 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01y2k57fs0kll514yf4m3hrt1gf60 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01s5q4lhr54afz1hllld32c9og9164 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0w3p8j1o8qxaag2n0h8vl04oq34 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0kj5s0ijujge51x8gezhmaezzr69 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01l02b3j987cp4zsjjbrpukwpe151 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09pv184jweyxt1v66h4p9cnrk946 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node015t8bhsnqup9u5wmbu80hcaez80 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0gjhsnygy1twa17l6pv6at5cvx175 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011gf3kboxcjt017nl7a5wuhq5u67 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0fb4k5tka2degu0xubwo6cwu331 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node017bqp1dra4c1q1epzic1pced1w39 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01j2wc3j1wyk1m1xv9z1x403wr466 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011lzjflafeb2g1f43y7dqha7pc147 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0cx08mvporo7y31gx5psp3txq131 2021-05-28T00:50:08.152Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node014k7rfdi04newx1nbeuos23qa157 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01fepgv8q8tmk3noisyj0p416a4 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02p9hn4utz80746ksp328w0hp118 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node04zwn8iwm0zaj1786cd7zozmna21 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node014m028vbyv24y1irxe9l4esyjh126 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0f7xocaxjgjfal1te6fjgy7n251 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node05jjhv0fm6a6d1pcldsugimd1p124 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09qovf74pwylw1nt5jcl242ite153 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01t2ukowvp4vjpm0hywovv16o343 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01s21omssh4igp77n6rcrsrsvi150 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0azcbgw313g36h0yukcgpa5lc163 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mrtc39qmc1z81wmgjgr7nct9i45 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node07hqq8z0k0nhsv4z9rop6hvzm57 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01bb6lffy0kf0qap6s3z9bjv52181 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0bpehxefppi0x1dxu3tgn8wmdl103 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0g312tetr48181f6w789kg8bs4107 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0s7rybmp3jz4j1601tyyoh9uwf79 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yzyc9kwtpfyf1she66qd1mu9x102 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01fea79f81bvgrd20gerlxmgkr121 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09p41bqfgg9wm1uj0unve5612r40 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ihkda994yb8sfabsx66rkygf100 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node07zqlqxgd0q2b1a1slp0fpg16792 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kf3wz7fj7ksq1mxqzck88edw2141 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0x6lx7coknosovnmr6zodlavq145 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0r7g3kefribls1elz3yqu88gtd38 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01e3w13qyugx3syfkya3yw25q6170 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01uuqimy19kt9h3q3984ukvme50 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node08w6hvm1s8dfn1rflsa8ytrqv7111 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node014ix5bdcr0bcp1w2tft0xsgnof171 2021-05-28T00:50:08.153Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0woxuxgdkasgsrm4wshd5gt5j72 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01lw1fftyovuzs1dysmak89b5f754 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node017f8v8p4wfegqm99vut8ku32e144 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01mq96w8eeqc1i1fpuznw0w5xux71 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0101sz6yn6fu7zp1s66zk1uksq2 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01m7oefi0p8vlo1ie48va9c68qt125 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0rj10p768bea9fgcfkv56xjuh127 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01704n98792ztu1cpiduq8ei39b139 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0qajeyk7sp2xf1upvue2afak8973 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0b3z22u2bgwab1ujhnngpto7mz96 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node08vp5l1xciqtydzl88egvs6ue173 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node05rbyp420ig3e1hr1di081th4w87 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node091hrrp0dbe575v2dell5z2oq94 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0flju6b1kogie10vbag10glptx62 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01exax2l2dv1lhvpfbhnyifjsh76 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0r0yj9nwaur8c1e95veo07caii137 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0a0nth0kvqkvhmwsrb9q0j9k7140 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0hesj3jrvot4w1wk9reyo218np167 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0191me0w4q1ixfphrkhnoh29ox91 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01nd0jwsuicm3p1pftczsn6cw686 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01wofu1ljdflzj1tffef9ets87793 2021-05-28T00:50:08.154Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0grdydm0ca2991kqbndxq01385120 2021-05-28T00:50:11.062Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:50:11.062Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:50:11.063Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:11.063Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7300872a-d14e-446b-ab21-87f993c1cd3f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:12.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051188d9-a237-49f9-9512-710c94b5fba9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:12.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051188d9-a237-49f9-9512-710c94b5fba9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d99b04d 2021-05-28T00:50:12.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051188d9-a237-49f9-9512-710c94b5fba9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:12.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051188d9-a237-49f9-9512-710c94b5fba9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:13.565Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=051188d9-a237-49f9-9512-710c94b5fba9 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:13.566Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=051188d9-a237-49f9-9512-710c94b5fba9 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:14.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2085a9-731d-4ed9-ab63-0a4fc4bcffd7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:14.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2085a9-731d-4ed9-ab63-0a4fc4bcffd7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@757f13f8 2021-05-28T00:50:14.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2085a9-731d-4ed9-ab63-0a4fc4bcffd7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:14.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2085a9-731d-4ed9-ab63-0a4fc4bcffd7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:16.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc181651-eb11-4dd5-adbe-d8f2393c7f19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:16.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc181651-eb11-4dd5-adbe-d8f2393c7f19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a0c1af2 2021-05-28T00:50:16.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc181651-eb11-4dd5-adbe-d8f2393c7f19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:16.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc181651-eb11-4dd5-adbe-d8f2393c7f19 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:18.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=525230d7-1910-45fd-a583-e84529aae265 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:18.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=525230d7-1910-45fd-a583-e84529aae265 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a47cc26 2021-05-28T00:50:18.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=525230d7-1910-45fd-a583-e84529aae265 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:18.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=525230d7-1910-45fd-a583-e84529aae265 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:19.084Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a17eb6ee-a781-4cf5-a9a9-c86bbd97eb71 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:19.084013Z InvocationID=e153daa0-3718-47ef-a811-b9ef64d67773 LogTimestamp=2021-05-28T00:50:19.084184Z Entering 2021-05-28T00:50:19.084Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a17eb6ee-a781-4cf5-a9a9-c86bbd97eb71 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:19.084013Z InvocationID=e153daa0-3718-47ef-a811-b9ef64d67773 LogTimestamp=2021-05-28T00:50:19.084184Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:50:19.361Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=276 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:50:19.085Z InvocationID=78357fae-d9e6-4d7f-b7bc-f9e1df6514b0 LogTimestamp=2021-05-28T00:50:19.361025Z ResponseDesc=OK Exiting. 2021-05-28T00:50:19.361Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:50:19.378Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=184551 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:47:14.827Z InvocationID=051659bd-d96a-4013-8a89-193d81d71792 ErrorCode=500 LogTimestamp=2021-05-28T00:50:19.378764Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:19.379Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:50:19.475Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5471a6b5-9581-4f44-931e-9a6c2dd6c64a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:19.475767Z InvocationID=a05fe5b9-bacf-47ce-adf3-693e341c7b06 LogTimestamp=2021-05-28T00:50:19.475821Z Entering 2021-05-28T00:50:19.476Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5471a6b5-9581-4f44-931e-9a6c2dd6c64a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:19.475767Z InvocationID=a05fe5b9-bacf-47ce-adf3-693e341c7b06 LogTimestamp=2021-05-28T00:50:19.475821Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:50:19.585Z [qtp580718781-12] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:50:19.476Z InvocationID=b3fe5b60-6d72-4e71-a667-93d0c3161b59 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@4eada2fc', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 107ms 2021-05-28T00:50:19.586Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=110 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:50:19.476Z InvocationID=b3fe5b60-6d72-4e71-a667-93d0c3161b59 ErrorCode=409 LogTimestamp=2021-05-28T00:50:19.586556Z ResponseDesc=Conflict Exiting. 2021-05-28T00:50:19.587Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:50:19.673Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:19.673891Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:19.674Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:19.673891Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:19.674Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:50:19.880Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=734b8529-a439-430f-830a-fdfb6886c5ef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:19.879864Z InvocationID=058abdb7-315c-4e90-87bc-079b2120fa80 LogTimestamp=2021-05-28T00:50:19.879912Z Entering 2021-05-28T00:50:19.880Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=734b8529-a439-430f-830a-fdfb6886c5ef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:19.879864Z InvocationID=058abdb7-315c-4e90-87bc-079b2120fa80 LogTimestamp=2021-05-28T00:50:19.879912Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:50:20.072Z [qtp580718781-96] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:50:19.880Z InvocationID=57282e0b-13a1-42a8-b40a-f32416f69de2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@4eada2fc', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 191ms 2021-05-28T00:50:20.073Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=193 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:50:19.880Z InvocationID=57282e0b-13a1-42a8-b40a-f32416f69de2 ErrorCode=400 LogTimestamp=2021-05-28T00:50:20.073335Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:50:20.073Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:50:20.163Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:20.163739Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:20.163Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:20.163739Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:20.164Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:50:20.172Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1bbc41dc-d0f9-4ab7-a78b-0df5c497a307 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:20.172711Z InvocationID=def24b1a-b4e4-450d-9743-52f86d5e5b85 LogTimestamp=2021-05-28T00:50:20.172741Z Entering 2021-05-28T00:50:20.172Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1bbc41dc-d0f9-4ab7-a78b-0df5c497a307 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:20.172711Z InvocationID=def24b1a-b4e4-450d-9743-52f86d5e5b85 LogTimestamp=2021-05-28T00:50:20.172741Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:50:20.378Z [qtp580718781-98] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:50:20.173Z InvocationID=891a3df7-bca2-4667-935a-286cd121d286 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@4eada2fc', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 205ms 2021-05-28T00:50:20.379Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=206 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:50:20.173Z InvocationID=891a3df7-bca2-4667-935a-286cd121d286 ErrorCode=400 LogTimestamp=2021-05-28T00:50:20.379589Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:50:20.380Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:50:20.463Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=65633 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:49:14.830Z InvocationID=4f892ac9-40fe-491d-b5eb-51cd1b731c6d ErrorCode=500 LogTimestamp=2021-05-28T00:50:20.463022Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:20.463Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:50:20.475Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0cc36062-33b0-4ec2-8f64-acafc62e711c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:20.475039Z InvocationID=6154e5db-b776-4c37-a448-6749e3347141 LogTimestamp=2021-05-28T00:50:20.475072Z Entering 2021-05-28T00:50:20.475Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0cc36062-33b0-4ec2-8f64-acafc62e711c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:20.475039Z InvocationID=6154e5db-b776-4c37-a448-6749e3347141 LogTimestamp=2021-05-28T00:50:20.475072Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:50:20.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79af4b16-fff8-4c05-b68a-b8905b58f5e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:20.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79af4b16-fff8-4c05-b68a-b8905b58f5e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34f9cd14 2021-05-28T00:50:20.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79af4b16-fff8-4c05-b68a-b8905b58f5e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:20.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79af4b16-fff8-4c05-b68a-b8905b58f5e6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:50:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:50:20.675Z [qtp580718781-16] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:50:20.475Z InvocationID=72cacc45-4c15-411a-9886-7724d96dd022 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@4eada2fc', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 199ms 2021-05-28T00:50:20.676Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=201 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:50:20.475Z InvocationID=72cacc45-4c15-411a-9886-7724d96dd022 ErrorCode=409 LogTimestamp=2021-05-28T00:50:20.676263Z ResponseDesc=Conflict Exiting. 2021-05-28T00:50:20.676Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:50:20.776Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:20.776839Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:20.776Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:20.776839Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:20.777Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:50:21.058Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3b831f32-d504-4733-9ca9-9cd3094357db InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:21.058044Z InvocationID=e21fcab5-a5ec-4e40-a4a1-d10acf756b6e LogTimestamp=2021-05-28T00:50:21.058090Z Entering 2021-05-28T00:50:21.058Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3b831f32-d504-4733-9ca9-9cd3094357db InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:21.058044Z InvocationID=e21fcab5-a5ec-4e40-a4a1-d10acf756b6e LogTimestamp=2021-05-28T00:50:21.058090Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:50:21.378Z [qtp580718781-15] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:50:21.058Z InvocationID=b0e43284-ab5c-489d-b2ca-f1d4b3fc8429 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@4eada2fc', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 318ms 2021-05-28T00:50:21.379Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=321 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:50:21.058Z InvocationID=b0e43284-ab5c-489d-b2ca-f1d4b3fc8429 ErrorCode=409 LogTimestamp=2021-05-28T00:50:21.379050Z ResponseDesc=Conflict Exiting. 2021-05-28T00:50:21.379Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:50:21.757Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:21.757636Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:21.757Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:21.757636Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:21.758Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:50:21.880Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=25640407-92fa-4764-b523-46dfe39e48f4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:21.880020Z InvocationID=c09de4cc-f3e8-486f-a728-0b7f76b9f22c LogTimestamp=2021-05-28T00:50:21.880054Z Entering 2021-05-28T00:50:21.880Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=25640407-92fa-4764-b523-46dfe39e48f4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:21.880020Z InvocationID=c09de4cc-f3e8-486f-a728-0b7f76b9f22c LogTimestamp=2021-05-28T00:50:21.880054Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:50:22.163Z [qtp580718781-11] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:50:21.880Z InvocationID=c80c9767-21ba-4b40-aeec-52db1b163b1d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@4eada2fc', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 281ms 2021-05-28T00:50:22.164Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=284 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:50:21.880Z InvocationID=c80c9767-21ba-4b40-aeec-52db1b163b1d ErrorCode=409 LogTimestamp=2021-05-28T00:50:22.164019Z ResponseDesc=Conflict Exiting. 2021-05-28T00:50:22.164Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:50:22.578Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fd23624f-f11f-4a05-b34b-df7e78110238 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:22.578413Z InvocationID=8c1d3639-b8ee-49bd-a306-237b1a8a8852 LogTimestamp=2021-05-28T00:50:22.578447Z Entering 2021-05-28T00:50:22.578Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fd23624f-f11f-4a05-b34b-df7e78110238 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:22.578413Z InvocationID=8c1d3639-b8ee-49bd-a306-237b1a8a8852 LogTimestamp=2021-05-28T00:50:22.578447Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:50:22.665Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=86 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:50:22.579Z InvocationID=e67f4e24-0b80-4229-90dc-d14a298030fc LogTimestamp=2021-05-28T00:50:22.665667Z ResponseDesc=OK Exiting. 2021-05-28T00:50:22.666Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:50:22.768Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:22.768033Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:22.768Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:22.768033Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:22.768Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:50:23.056Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8873d036-e093-4be1-b6d9-57f3530c9e85 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:23.056705Z InvocationID=ed5bfd79-122e-4216-88bc-2eeee79970b7 LogTimestamp=2021-05-28T00:50:23.056747Z Entering 2021-05-28T00:50:23.057Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8873d036-e093-4be1-b6d9-57f3530c9e85 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:23.056705Z InvocationID=ed5bfd79-122e-4216-88bc-2eeee79970b7 LogTimestamp=2021-05-28T00:50:23.056747Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:23.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8772787-af33-4bd4-a94e-c8bf4f2ef19f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:23.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8772787-af33-4bd4-a94e-c8bf4f2ef19f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3914f8a0 2021-05-28T00:50:23.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8772787-af33-4bd4-a94e-c8bf4f2ef19f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:23.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8772787-af33-4bd4-a94e-c8bf4f2ef19f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:23.158Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:23.057Z InvocationID=973c5b9f-7419-4172-ab80-6b0174c247d4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:50:24.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44d2ae9-2f98-4fee-933c-96af18527163 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:24.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44d2ae9-2f98-4fee-933c-96af18527163 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7305aefa 2021-05-28T00:50:24.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44d2ae9-2f98-4fee-933c-96af18527163 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:24.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d44d2ae9-2f98-4fee-933c-96af18527163 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:26.674Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3616 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:23.057Z InvocationID=973c5b9f-7419-4172-ab80-6b0174c247d4 LogTimestamp=2021-05-28T00:50:26.673863Z ResponseDesc=Created Exiting. 2021-05-28T00:50:26.675Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:50:26.857Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:26.857190Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:26.857Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:26.857190Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:26.857Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:50:26.880Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=56a5d481-8027-4509-bd31-b70dfbf3b85e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:26.880109Z InvocationID=6a36964d-18de-452e-ac1e-e6992a4f411f LogTimestamp=2021-05-28T00:50:26.880149Z Entering 2021-05-28T00:50:26.880Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=56a5d481-8027-4509-bd31-b70dfbf3b85e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:26.880109Z InvocationID=6a36964d-18de-452e-ac1e-e6992a4f411f LogTimestamp=2021-05-28T00:50:26.880149Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:26.973Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:26.880Z InvocationID=55a2cc17-528b-4edc-8a23-671db340447a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:50:27.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f4cfea-f6b0-4f4e-8e04-314450981fb7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:27.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f4cfea-f6b0-4f4e-8e04-314450981fb7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37e76a40 2021-05-28T00:50:27.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f4cfea-f6b0-4f4e-8e04-314450981fb7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:27.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f4cfea-f6b0-4f4e-8e04-314450981fb7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:29.775Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2894 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:26.880Z InvocationID=55a2cc17-528b-4edc-8a23-671db340447a LogTimestamp=2021-05-28T00:50:29.774854Z ResponseDesc=Created Exiting. 2021-05-28T00:50:29.775Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:50:29.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=257bf88e-0ca2-422d-a8cb-b4d942434196 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:29.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=257bf88e-0ca2-422d-a8cb-b4d942434196 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d8c90c5 2021-05-28T00:50:29.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=257bf88e-0ca2-422d-a8cb-b4d942434196 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:29.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=257bf88e-0ca2-422d-a8cb-b4d942434196 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:29.978Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:29.978587Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:29.978Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:29.978587Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:29.979Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:50:30.171Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8e3219af-650e-41ce-9b40-392b15bcb3ee InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:30.171030Z InvocationID=220f00e7-e179-4275-be28-a0d89eec1333 LogTimestamp=2021-05-28T00:50:30.171065Z Entering 2021-05-28T00:50:30.171Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8e3219af-650e-41ce-9b40-392b15bcb3ee InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:30.171030Z InvocationID=220f00e7-e179-4275-be28-a0d89eec1333 LogTimestamp=2021-05-28T00:50:30.171065Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:30.356Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:30.171Z InvocationID=e55c4c87-0c45-4467-8bc7-df98800c5d7a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:50:31.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866dc9bc-3632-4d3f-aaf2-b29925beadaa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:31.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866dc9bc-3632-4d3f-aaf2-b29925beadaa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53c3a02c 2021-05-28T00:50:31.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866dc9bc-3632-4d3f-aaf2-b29925beadaa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:31.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866dc9bc-3632-4d3f-aaf2-b29925beadaa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:32.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9928ad8-06f5-44f7-9e83-b681d2299422 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:32.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9928ad8-06f5-44f7-9e83-b681d2299422 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d6fa6d0 2021-05-28T00:50:32.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9928ad8-06f5-44f7-9e83-b681d2299422 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:32.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9928ad8-06f5-44f7-9e83-b681d2299422 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:33.882Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3711 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:30.171Z InvocationID=e55c4c87-0c45-4467-8bc7-df98800c5d7a LogTimestamp=2021-05-28T00:50:33.882098Z ResponseDesc=Created Exiting. 2021-05-28T00:50:33.882Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:50:33.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:33.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d66857 2021-05-28T00:50:33.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:33.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:34.171Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:34.171181Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:34.171Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:34.171181Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:34.172Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:50:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:50:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:50:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:50:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:50:36.967Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:50:36.968Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:50:36.968Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:36.968Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=28a39fc7-0081-4a84-ba22-b6d9ade42e4a ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:50:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:50:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:50:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:50:39.669Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=88010557-dc07-4125-b1cd-a3c921e8cae2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:39.668879Z InvocationID=207683fd-2c00-4010-9ab5-300e7c6d118c LogTimestamp=2021-05-28T00:50:39.668951Z Entering 2021-05-28T00:50:39.669Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=88010557-dc07-4125-b1cd-a3c921e8cae2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:39.668879Z InvocationID=207683fd-2c00-4010-9ab5-300e7c6d118c LogTimestamp=2021-05-28T00:50:39.668951Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:39.670Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-05-28T00:50:39.670Z InvocationID=890fd277-36cc-40ae-a496-d4d5a2488987 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:50:39.671Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:50:39.670Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:39.670Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=890fd277-36cc-40ae-a496-d4d5a2488987 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@58b5fa0a 2021-05-28T00:50:39.671Z [qtp580718781-18] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:50:39.670Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:39.670Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=890fd277-36cc-40ae-a496-d4d5a2488987 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@58b5fa0a 2021-05-28T00:50:39.671Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:39.671470Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:39.671Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:39.671470Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:39.672Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="500" 2021-05-28T00:50:39.873Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:39.873676Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:39.873Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:39.873676Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:39.874Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:50:39.973Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4b020056-35c7-4dda-b2e9-ae241d1b1db1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:39.973436Z InvocationID=93d6b092-b74e-4664-be33-4bf9733aa4a8 LogTimestamp=2021-05-28T00:50:39.973478Z Entering 2021-05-28T00:50:39.973Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4b020056-35c7-4dda-b2e9-ae241d1b1db1 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:39.973436Z InvocationID=93d6b092-b74e-4664-be33-4bf9733aa4a8 LogTimestamp=2021-05-28T00:50:39.973478Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:39.974Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-05-28T00:50:39.973Z InvocationID=1523b92a-93ad-472d-b6b2-ee6f62786a8c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:50:39.974Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:50:39.974Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:39.973Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=1523b92a-93ad-472d-b6b2-ee6f62786a8c ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@793aef1a 2021-05-28T00:50:39.974Z [qtp580718781-98] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:50:39.974Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:39.973Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=1523b92a-93ad-472d-b6b2-ee6f62786a8c ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@793aef1a 2021-05-28T00:50:39.975Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:39.974998Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:39.975Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:39.974998Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:39.975Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="500" 2021-05-28T00:50:40.264Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.264115Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:40.264Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.264115Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:40.264Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:50:40.370Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4307e453-1553-4a80-9515-8592374bc1cb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:40.370078Z InvocationID=284b1e22-222c-4682-8740-1dbc35bd2a73 LogTimestamp=2021-05-28T00:50:40.370122Z Entering 2021-05-28T00:50:40.370Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4307e453-1553-4a80-9515-8592374bc1cb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:40.370078Z InvocationID=284b1e22-222c-4682-8740-1dbc35bd2a73 LogTimestamp=2021-05-28T00:50:40.370122Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:40.370Z [qtp580718781-16] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-05-28T00:50:40.370Z InvocationID=81f60381-a09c-440e-8ef0-668d44f70ca3 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:50:40.371Z [qtp580718781-16] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:50:40.370Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:40.370Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=81f60381-a09c-440e-8ef0-668d44f70ca3 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@38ba474d 2021-05-28T00:50:40.371Z [qtp580718781-16] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:50:40.370Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:40.370Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=81f60381-a09c-440e-8ef0-668d44f70ca3 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@38ba474d 2021-05-28T00:50:40.371Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.371137Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:40.371Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.371137Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:40.371Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="500" 2021-05-28T00:50:40.483Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.483361Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:40.483Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.483361Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:40.483Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:50:40.761Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f31ee968-7dd2-419f-a9f2-c5eb88dea10b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:40.761817Z InvocationID=316f440d-000e-40a1-8e97-ae866fb3fb92 LogTimestamp=2021-05-28T00:50:40.761860Z Entering 2021-05-28T00:50:40.762Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f31ee968-7dd2-419f-a9f2-c5eb88dea10b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:40.761817Z InvocationID=316f440d-000e-40a1-8e97-ae866fb3fb92 LogTimestamp=2021-05-28T00:50:40.761860Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:40.762Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-05-28T00:50:40.762Z InvocationID=12907aba-5a09-43e3-8445-ecb8fa5cd1b6 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:50:40.762Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:50:40.762Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:40.762Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=12907aba-5a09-43e3-8445-ecb8fa5cd1b6 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@754cb18f 2021-05-28T00:50:40.762Z [qtp580718781-17] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:50:40.762Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:50:40.762Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=12907aba-5a09-43e3-8445-ecb8fa5cd1b6 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@754cb18f 2021-05-28T00:50:40.762Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.762834Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:40.762Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.762834Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:40.763Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="500" 2021-05-28T00:50:40.867Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:50:40.869Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.869753Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:40.870Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:40.869753Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:40.870Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:50:40.971Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7ae8b978-bbbb-43eb-a655-9592e1c4a990 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:40.971479Z InvocationID=c1d43427-3476-49bb-b995-74969e13656e LogTimestamp=2021-05-28T00:50:40.971522Z Entering 2021-05-28T00:50:40.971Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7ae8b978-bbbb-43eb-a655-9592e1c4a990 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:40.971479Z InvocationID=c1d43427-3476-49bb-b995-74969e13656e LogTimestamp=2021-05-28T00:50:40.971522Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:41.070Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:40.971Z InvocationID=217066cb-6656-48d3-b4e0-ba683067763f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:50:41.975Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:50:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eafc62c-2b1c-4525-9a8a-06ad7ab3f54c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eafc62c-2b1c-4525-9a8a-06ad7ab3f54c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2875ca10 2021-05-28T00:50:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eafc62c-2b1c-4525-9a8a-06ad7ab3f54c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:42.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eafc62c-2b1c-4525-9a8a-06ad7ab3f54c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:43.470Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0eafc62c-2b1c-4525-9a8a-06ad7ab3f54c ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:43.470Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=0eafc62c-2b1c-4525-9a8a-06ad7ab3f54c ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:50:44.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea894646-a22e-4656-bc6b-964af37bc72f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:44.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea894646-a22e-4656-bc6b-964af37bc72f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1650bedb 2021-05-28T00:50:44.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea894646-a22e-4656-bc6b-964af37bc72f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:44.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea894646-a22e-4656-bc6b-964af37bc72f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:50:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:50:47.881Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ea894646-a22e-4656-bc6b-964af37bc72f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:50:47.882Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ea894646-a22e-4656-bc6b-964af37bc72f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:50:47.882Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ea894646-a22e-4656-bc6b-964af37bc72f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:47.882Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ea894646-a22e-4656-bc6b-964af37bc72f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:50:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:50:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:50:52.195Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:50:52.957Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:50:52.969Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:50:53.072Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:50:53.460Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=12489 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:40.971Z InvocationID=217066cb-6656-48d3-b4e0-ba683067763f LogTimestamp=2021-05-28T00:50:53.460438Z ResponseDesc=Created Exiting. 2021-05-28T00:50:53.461Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:50:53.472Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:53.472622Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:53.472Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:53.472622Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:53.473Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:50:53.572Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=25c9fd87-d340-4867-9b19-858194f06350 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:53.572189Z InvocationID=1d2e07c2-a1e2-4aab-b0fd-12d9b4d1ae95 LogTimestamp=2021-05-28T00:50:53.572233Z Entering 2021-05-28T00:50:53.572Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=25c9fd87-d340-4867-9b19-858194f06350 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:53.572189Z InvocationID=1d2e07c2-a1e2-4aab-b0fd-12d9b4d1ae95 LogTimestamp=2021-05-28T00:50:53.572233Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:53.763Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:53.572Z InvocationID=d399499c-ed5a-427d-8cbb-831855871362 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:50:55.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565fa012-95a7-4649-9920-2fadc005f18d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:55.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565fa012-95a7-4649-9920-2fadc005f18d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24f6e896 2021-05-28T00:50:55.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565fa012-95a7-4649-9920-2fadc005f18d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:55.176Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=565fa012-95a7-4649-9920-2fadc005f18d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:50:56.374Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2802 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:53.572Z InvocationID=d399499c-ed5a-427d-8cbb-831855871362 LogTimestamp=2021-05-28T00:50:56.374293Z ResponseDesc=Created Exiting. 2021-05-28T00:50:56.374Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:50:56.383Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:56.383555Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:50:56.383Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:50:56.383555Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:50:56.384Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:50:56.570Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7b873738-18a4-4ce5-aaa7-c18d61631f0e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:56.570537Z InvocationID=37329d1d-5a79-4cf1-81a6-f06bb4ceee19 LogTimestamp=2021-05-28T00:50:56.570590Z Entering 2021-05-28T00:50:56.570Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7b873738-18a4-4ce5-aaa7-c18d61631f0e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:50:56.570537Z InvocationID=37329d1d-5a79-4cf1-81a6-f06bb4ceee19 LogTimestamp=2021-05-28T00:50:56.570590Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:50:56.578Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:56.571Z InvocationID=8de22473-a28e-41bc-845c-2570d3238f3e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:50:57.164Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=565fa012-95a7-4649-9920-2fadc005f18d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:50:57.164Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=565fa012-95a7-4649-9920-2fadc005f18d ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:50:58.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:50:58.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a6b5cb7 2021-05-28T00:50:58.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:50:58.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:00.687Z [qtp580718781-19] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:56.571Z InvocationID=8de22473-a28e-41bc-845c-2570d3238f3e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Temporary storage failure while checking locks org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.getSliceWithRetries(ConsistentKeyLocker.java:464) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:426) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.checkSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.checkLocks(AbstractLocker.java:351) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.checkAllLocks(ExpectedValueCheckingTransaction.java:175) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.prepareForMutations(ExpectedValueCheckingTransaction.java:154) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStoreManager.mutateMany(ExpectedValueCheckingStoreManager.java:72) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:94) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction$1.call(CacheTransaction.java:91) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.persist(CacheTransaction.java:91) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.flushInternal(CacheTransaction.java:139) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:196) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:279) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1265) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1249) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:52) at org.openecomp.sdc.be.components.impl.ConsumerBusinessLogic.createConsumer(ConsumerBusinessLogic.java:93) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer_aroundBody0(ConsumerServlet.java:115) at org.openecomp.sdc.be.servlets.ConsumerServlet$AjcClosure1.run(ConsumerServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ConsumerServlet.createConsumer(ConsumerServlet.java:96) at jdk.internal.reflect.GeneratedMethodAccessor93.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:75) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:76) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.exceptions.ReadTimeoutException.copy(ReadTimeoutException.java:124) at com.datastax.driver.core.Responses$Error.asException(Responses.java:169) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:645) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.ReadTimeoutException: Cassandra timeout during read query at consistency ALL (3 responses were required but only 2 replica responded) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:90) at com.datastax.driver.core.Responses$Error$1.decode(Responses.java:65) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:297) at com.datastax.driver.core.Message$ProtocolDecoder.decode(Message.java:268) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ... 24 common frames omitted 2021-05-28T00:51:01.580Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:51:01.580Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:51:01.580Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:01.580Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=329873f6-21f4-4fbf-82b5-b91b9f514425 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a870433-b364-45dd-83b4-7192f9cd0766 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a870433-b364-45dd-83b4-7192f9cd0766 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19de8d47 2021-05-28T00:51:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a870433-b364-45dd-83b4-7192f9cd0766 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:02.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a870433-b364-45dd-83b4-7192f9cd0766 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:51:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:51:05.552Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=6a870433-b364-45dd-83b4-7192f9cd0766 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:51:05.552Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=6a870433-b364-45dd-83b4-7192f9cd0766 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:51:07.267Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=10695 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:50:56.571Z InvocationID=8de22473-a28e-41bc-845c-2570d3238f3e LogTimestamp=2021-05-28T00:51:07.266558Z ResponseDesc=Created Exiting. 2021-05-28T00:51:07.267Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:51:07.375Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:07.374910Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:51:07.375Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:07.374910Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:07.376Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:51:07.467Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b2b8a737-0c76-4c5f-a35d-555dc5b4da20 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:07.467275Z InvocationID=a60a13e2-2c7f-4602-9663-b7ca9c25d047 LogTimestamp=2021-05-28T00:51:07.467318Z Entering 2021-05-28T00:51:07.467Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b2b8a737-0c76-4c5f-a35d-555dc5b4da20 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:07.467275Z InvocationID=a60a13e2-2c7f-4602-9663-b7ca9c25d047 LogTimestamp=2021-05-28T00:51:07.467318Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:51:07.490Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:07.467Z InvocationID=b2cd9a2f-47af-4137-a369-763ceba31486 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:51:07.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a7c2b9-eec2-425e-87c0-6eecad9b9c46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:07.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a7c2b9-eec2-425e-87c0-6eecad9b9c46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43ea9ac 2021-05-28T00:51:07.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a7c2b9-eec2-425e-87c0-6eecad9b9c46 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:07.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a7c2b9-eec2-425e-87c0-6eecad9b9c46 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:09.989Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=35a7c2b9-eec2-425e-87c0-6eecad9b9c46 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:09.989Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=35a7c2b9-eec2-425e-87c0-6eecad9b9c46 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:10.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98d565eb-a8e0-46b4-a027-61a51673d07d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:10.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98d565eb-a8e0-46b4-a027-61a51673d07d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39b9d806 2021-05-28T00:51:10.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98d565eb-a8e0-46b4-a027-61a51673d07d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:10.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98d565eb-a8e0-46b4-a027-61a51673d07d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:11.678Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4211 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:07.467Z InvocationID=b2cd9a2f-47af-4137-a369-763ceba31486 LogTimestamp=2021-05-28T00:51:11.678030Z ResponseDesc=Created Exiting. 2021-05-28T00:51:11.678Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:51:12.071Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:12.071414Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:51:12.071Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:12.071414Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:12.072Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:51:12.171Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bec49b6b-7a68-41a1-b4b6-19b8705b79e2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:12.170972Z InvocationID=6ee1db56-93b2-4666-b450-0703ca46fbd3 LogTimestamp=2021-05-28T00:51:12.171048Z Entering 2021-05-28T00:51:12.171Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bec49b6b-7a68-41a1-b4b6-19b8705b79e2 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:12.170972Z InvocationID=6ee1db56-93b2-4666-b450-0703ca46fbd3 LogTimestamp=2021-05-28T00:51:12.171048Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:51:12.263Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:12.171Z InvocationID=914261b6-b669-4c6f-ac4f-e354c9066593 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:51:12.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e23305fe-a48c-4287-9fcc-714588dab6b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:12.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e23305fe-a48c-4287-9fcc-714588dab6b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4812f167 2021-05-28T00:51:12.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e23305fe-a48c-4287-9fcc-714588dab6b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:12.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e23305fe-a48c-4287-9fcc-714588dab6b1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:14.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7054c6ca-f2de-456c-81e7-e6c3c82191bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:14.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7054c6ca-f2de-456c-81e7-e6c3c82191bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39905200 2021-05-28T00:51:14.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7054c6ca-f2de-456c-81e7-e6c3c82191bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:14.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7054c6ca-f2de-456c-81e7-e6c3c82191bf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:15.278Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3107 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:12.171Z InvocationID=914261b6-b669-4c6f-ac4f-e354c9066593 LogTimestamp=2021-05-28T00:51:15.278423Z ResponseDesc=Created Exiting. 2021-05-28T00:51:15.279Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:51:15.879Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:15.879622Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:51:15.879Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:15.879622Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:15.880Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:51:16.076Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ff0531bd-3414-432c-86fc-f0a9b4318443 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:16.076107Z InvocationID=234782aa-057f-4e88-a0d5-829e0ceb0942 LogTimestamp=2021-05-28T00:51:16.076159Z Entering 2021-05-28T00:51:16.076Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ff0531bd-3414-432c-86fc-f0a9b4318443 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:16.076107Z InvocationID=234782aa-057f-4e88-a0d5-829e0ceb0942 LogTimestamp=2021-05-28T00:51:16.076159Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:51:16.083Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:16.076Z InvocationID=1cb9fed7-d5a7-4f2f-96bc-11c8483595a9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:51:16.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a0ef23-336d-4ec7-85fb-7c641164aa51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:16.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a0ef23-336d-4ec7-85fb-7c641164aa51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b392d49 2021-05-28T00:51:16.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a0ef23-336d-4ec7-85fb-7c641164aa51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:16.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a0ef23-336d-4ec7-85fb-7c641164aa51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:18.201Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1611eb00-0b0b-4c18-8d5b-dcfd107f1af6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:18.201Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1611eb00-0b0b-4c18-8d5b-dcfd107f1af6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c37a1c0 2021-05-28T00:51:18.201Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1611eb00-0b0b-4c18-8d5b-dcfd107f1af6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:18.201Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1611eb00-0b0b-4c18-8d5b-dcfd107f1af6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:19.075Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2999 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:16.076Z InvocationID=1cb9fed7-d5a7-4f2f-96bc-11c8483595a9 LogTimestamp=2021-05-28T00:51:19.075573Z ResponseDesc=Created Exiting. 2021-05-28T00:51:19.075Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:51:19.172Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:19.172669Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:51:19.172Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:19.172669Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:19.173Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:51:19.377Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=304db49e-592e-4c2b-9f42-16f28a2e947b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:19.377440Z InvocationID=f3d30234-c4c3-4605-9bbb-570c38e0be60 LogTimestamp=2021-05-28T00:51:19.377504Z Entering 2021-05-28T00:51:19.377Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=304db49e-592e-4c2b-9f42-16f28a2e947b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:19.377440Z InvocationID=f3d30234-c4c3-4605-9bbb-570c38e0be60 LogTimestamp=2021-05-28T00:51:19.377504Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:51:19.468Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:19.378Z InvocationID=1de99550-3728-49b7-944e-34fffddc7e29 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:51:20.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cde1f89-127b-4182-b22d-76476f9c5061 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:20.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cde1f89-127b-4182-b22d-76476f9c5061 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43f1e2e8 2021-05-28T00:51:20.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cde1f89-127b-4182-b22d-76476f9c5061 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:20.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cde1f89-127b-4182-b22d-76476f9c5061 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:51:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:51:21.668Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2290 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:19.378Z InvocationID=1de99550-3728-49b7-944e-34fffddc7e29 LogTimestamp=2021-05-28T00:51:21.668461Z ResponseDesc=Created Exiting. 2021-05-28T00:51:21.669Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:51:21.772Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:21.772121Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:51:21.772Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:21.772121Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:21.772Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:51:21.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d8bc204-904e-4d70-a8a4-e80665993506 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:21.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d8bc204-904e-4d70-a8a4-e80665993506 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@784aa8c4 2021-05-28T00:51:21.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d8bc204-904e-4d70-a8a4-e80665993506 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:21.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d8bc204-904e-4d70-a8a4-e80665993506 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:21.871Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=20027811-2144-426f-97d9-2c5984442b94 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:21.871588Z InvocationID=d3526f18-a221-4014-9d28-02e577e6b059 LogTimestamp=2021-05-28T00:51:21.871630Z Entering 2021-05-28T00:51:21.871Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=20027811-2144-426f-97d9-2c5984442b94 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:21.871588Z InvocationID=d3526f18-a221-4014-9d28-02e577e6b059 LogTimestamp=2021-05-28T00:51:21.871630Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:51:21.965Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:21.872Z InvocationID=10205235-cfd2-4b52-917e-7c36025384c3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:51:24.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae471e8-a71a-473f-97e3-bb4e630a59fa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:24.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae471e8-a71a-473f-97e3-bb4e630a59fa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a3b91e1 2021-05-28T00:51:24.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae471e8-a71a-473f-97e3-bb4e630a59fa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:24.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae471e8-a71a-473f-97e3-bb4e630a59fa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:25.057Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=3184 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:51:21.872Z InvocationID=10205235-cfd2-4b52-917e-7c36025384c3 LogTimestamp=2021-05-28T00:51:25.056892Z ResponseDesc=Created Exiting. 2021-05-28T00:51:25.057Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:51:25.660Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ae842d96-def6-42d3-a9b8-159758500d21 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:25.660025Z InvocationID=99133d16-a7f6-4308-b2eb-f44e422e81d2 LogTimestamp=2021-05-28T00:51:25.660098Z Entering 2021-05-28T00:51:25.660Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ae842d96-def6-42d3-a9b8-159758500d21 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:25.660025Z InvocationID=99133d16-a7f6-4308-b2eb-f44e422e81d2 LogTimestamp=2021-05-28T00:51:25.660098Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:51:25.857Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=197 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:51:25.660Z InvocationID=f2397881-0807-444a-9ca5-8d2ad390c11e LogTimestamp=2021-05-28T00:51:25.857725Z ResponseDesc=OK Exiting. 2021-05-28T00:51:25.858Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:51:26.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=021c2493-54c0-41cf-a6ef-659e26184494 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:26.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=021c2493-54c0-41cf-a6ef-659e26184494 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b2302c7 2021-05-28T00:51:26.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=021c2493-54c0-41cf-a6ef-659e26184494 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:26.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=021c2493-54c0-41cf-a6ef-659e26184494 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:26.472Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=71faec9e-8425-4ca7-b40b-8f4cbfe6c52f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:26.472516Z InvocationID=a4748b82-1f21-4c8d-b7b3-5917a6a72705 LogTimestamp=2021-05-28T00:51:26.472556Z Entering 2021-05-28T00:51:26.472Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=71faec9e-8425-4ca7-b40b-8f4cbfe6c52f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:26.472516Z InvocationID=a4748b82-1f21-4c8d-b7b3-5917a6a72705 LogTimestamp=2021-05-28T00:51:26.472556Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:51:26.971Z [qtp580718781-98] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:51:26.472Z InvocationID=bed2ca31-880d-4b2e-a030-cac4aae5ac6e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 495969563 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n495969563 100% \n 2021-05-28T00:51:26.979Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=507 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:51:26.472Z InvocationID=bed2ca31-880d-4b2e-a030-cac4aae5ac6e LogTimestamp=2021-05-28T00:51:26.979424Z ResponseDesc=OK Exiting. 2021-05-28T00:51:26.979Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:51:27.162Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e9b42d4f-31dc-4448-8e3d-cc8f71f5d428 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:27.162044Z InvocationID=8ec7aecd-f464-4689-9928-5c578a2883cd LogTimestamp=2021-05-28T00:51:27.162091Z Entering 2021-05-28T00:51:27.162Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e9b42d4f-31dc-4448-8e3d-cc8f71f5d428 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:27.162044Z InvocationID=8ec7aecd-f464-4689-9928-5c578a2883cd LogTimestamp=2021-05-28T00:51:27.162091Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:51:27.431Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:51:27.162Z InvocationID=e6934b84-fd89-499c-86f6-cc49d1132a5b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:51:27.505Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:51:27.162Z InvocationID=e6934b84-fd89-499c-86f6-cc49d1132a5b ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:51:27.506Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:51:27.162Z InvocationID=e6934b84-fd89-499c-86f6-cc49d1132a5b ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:51:27.506Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=344 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:51:27.162Z InvocationID=e6934b84-fd89-499c-86f6-cc49d1132a5b ErrorCode=500 LogTimestamp=2021-05-28T00:51:27.506423Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:27.506Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:51:27.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b79a6ad-b841-47dd-991c-b4afed609a9c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:27.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b79a6ad-b841-47dd-991c-b4afed609a9c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@378949e9 2021-05-28T00:51:27.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b79a6ad-b841-47dd-991c-b4afed609a9c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:27.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b79a6ad-b841-47dd-991c-b4afed609a9c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:29.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56fc8188-7a49-4089-a56e-109cb352ca79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:29.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56fc8188-7a49-4089-a56e-109cb352ca79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1aa8cb54 2021-05-28T00:51:29.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56fc8188-7a49-4089-a56e-109cb352ca79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:29.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56fc8188-7a49-4089-a56e-109cb352ca79 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:30.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:30.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c9ae579 2021-05-28T00:51:30.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:30.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:51:32.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:51:33.674Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:51:33.675Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:51:33.675Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:33.676Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b2072de6-46d6-4c6e-84c5-ed225710a752 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:35.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:51:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:51:35.519Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:51:36.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7343cb34-896b-4331-8310-65581af90232 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:36.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7343cb34-896b-4331-8310-65581af90232 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fccf917 2021-05-28T00:51:36.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7343cb34-896b-4331-8310-65581af90232 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:36.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7343cb34-896b-4331-8310-65581af90232 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:36.876Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7343cb34-896b-4331-8310-65581af90232 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:36.876Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=7343cb34-896b-4331-8310-65581af90232 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:51:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:51:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:51:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:51:37.575Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=23b3b4e7-1cd7-4f12-831f-fe5174a7b4ec InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:37.575041Z InvocationID=29702ba3-0824-4509-bc40-3ace7a87730f LogTimestamp=2021-05-28T00:51:37.575116Z Entering 2021-05-28T00:51:37.575Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=23b3b4e7-1cd7-4f12-831f-fe5174a7b4ec InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:37.575041Z InvocationID=29702ba3-0824-4509-bc40-3ace7a87730f LogTimestamp=2021-05-28T00:51:37.575116Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:51:37.575Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:51:37.575Z InvocationID=d699640c-729e-4648-81b8-10bbd78b1d52 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:51:37.576Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:51:37.576Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:51:37.575Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=d699640c-729e-4648-81b8-10bbd78b1d52 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@292245f3 2021-05-28T00:51:37.576Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T00:51:37.576Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:51:37.575Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=d699640c-729e-4648-81b8-10bbd78b1d52 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@292245f3 2021-05-28T00:51:37.576Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:37.576570Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:51:37.576Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:51:37.576570Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:37.577Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:51:37.578Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:51:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2ca81cc-2079-4395-a5d2-83bfe53f1c0c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2ca81cc-2079-4395-a5d2-83bfe53f1c0c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ed9a877 2021-05-28T00:51:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2ca81cc-2079-4395-a5d2-83bfe53f1c0c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2ca81cc-2079-4395-a5d2-83bfe53f1c0c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:40.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce44517-1bcb-47d7-98f4-10c7e3b621eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:40.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce44517-1bcb-47d7-98f4-10c7e3b621eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15232ff8 2021-05-28T00:51:40.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce44517-1bcb-47d7-98f4-10c7e3b621eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:40.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce44517-1bcb-47d7-98f4-10c7e3b621eb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:41.288Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=dce44517-1bcb-47d7-98f4-10c7e3b621eb ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:41.288Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=dce44517-1bcb-47d7-98f4-10c7e3b621eb ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:51:42.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78897647-d1eb-43f4-9bf2-d31bd39421db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:42.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78897647-d1eb-43f4-9bf2-d31bd39421db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@dbf780a 2021-05-28T00:51:42.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78897647-d1eb-43f4-9bf2-d31bd39421db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:42.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78897647-d1eb-43f4-9bf2-d31bd39421db PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:44.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e29c4b-25c2-43d9-b858-4ca82cd72b97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:44.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e29c4b-25c2-43d9-b858-4ca82cd72b97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36fb17aa 2021-05-28T00:51:44.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e29c4b-25c2-43d9-b858-4ca82cd72b97 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:44.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e29c4b-25c2-43d9-b858-4ca82cd72b97 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:46.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cab001-774f-4c70-882e-8e574266641f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:46.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cab001-774f-4c70-882e-8e574266641f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25729b12 2021-05-28T00:51:46.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cab001-774f-4c70-882e-8e574266641f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:46.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cab001-774f-4c70-882e-8e574266641f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:47.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72af8306-2a08-4fdc-a359-19a232b58883 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:47.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72af8306-2a08-4fdc-a359-19a232b58883 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29a22cab 2021-05-28T00:51:47.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72af8306-2a08-4fdc-a359-19a232b58883 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:47.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72af8306-2a08-4fdc-a359-19a232b58883 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:47.878Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ae968267-a3e5-43cf-8711-58ffeea0f874 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:47.878285Z InvocationID=d5902981-99f2-4509-819c-a6f1d676cf31 LogTimestamp=2021-05-28T00:51:47.878355Z Entering 2021-05-28T00:51:47.878Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ae968267-a3e5-43cf-8711-58ffeea0f874 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:47.878285Z InvocationID=d5902981-99f2-4509-819c-a6f1d676cf31 LogTimestamp=2021-05-28T00:51:47.878355Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:51:47.970Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:51:47.878Z InvocationID=de473a82-07ae-462f-8b64-07fb54959fbf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:51:48.065Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:51:47.878Z InvocationID=de473a82-07ae-462f-8b64-07fb54959fbf ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:51:48.065Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:51:47.878Z InvocationID=de473a82-07ae-462f-8b64-07fb54959fbf ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:51:48.066Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=187 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:51:47.878Z InvocationID=de473a82-07ae-462f-8b64-07fb54959fbf ErrorCode=500 LogTimestamp=2021-05-28T00:51:48.065977Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:48.066Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:51:49.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fab31d-7ed6-4456-a561-b585eec7bd6d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:49.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fab31d-7ed6-4456-a561-b585eec7bd6d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f2b6ebd 2021-05-28T00:51:49.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fab31d-7ed6-4456-a561-b585eec7bd6d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:49.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0fab31d-7ed6-4456-a561-b585eec7bd6d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:51:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:51:50.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6854e9f6-af2c-426a-b07b-8c1433db3c39 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:50.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6854e9f6-af2c-426a-b07b-8c1433db3c39 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@318364b5 2021-05-28T00:51:50.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6854e9f6-af2c-426a-b07b-8c1433db3c39 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:50.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6854e9f6-af2c-426a-b07b-8c1433db3c39 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:52.162Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:51:52.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f9dd9b5-d71d-447c-9261-53fc38b69b65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:52.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f9dd9b5-d71d-447c-9261-53fc38b69b65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@98d39c 2021-05-28T00:51:52.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f9dd9b5-d71d-447c-9261-53fc38b69b65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:52.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f9dd9b5-d71d-447c-9261-53fc38b69b65 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:53.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f35438b0-5929-42cb-ad29-1a8d395021e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:53.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f35438b0-5929-42cb-ad29-1a8d395021e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38e76050 2021-05-28T00:51:53.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f35438b0-5929-42cb-ad29-1a8d395021e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:53.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f35438b0-5929-42cb-ad29-1a8d395021e1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:56.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8cc2f7-38cd-47b7-9474-ac4da6cdba8f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:56.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8cc2f7-38cd-47b7-9474-ac4da6cdba8f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cbdaf77 2021-05-28T00:51:56.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8cc2f7-38cd-47b7-9474-ac4da6cdba8f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:56.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8cc2f7-38cd-47b7-9474-ac4da6cdba8f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:58.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c77eb22d-a3e4-4894-bd47-61e47e7348b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:51:58.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c77eb22d-a3e4-4894-bd47-61e47e7348b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16b1d299 2021-05-28T00:51:58.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c77eb22d-a3e4-4894-bd47-61e47e7348b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:51:58.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c77eb22d-a3e4-4894-bd47-61e47e7348b4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:51:58.275Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c3cc8a40-334a-40f8-a2ac-08e40b13a11b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:58.275215Z InvocationID=989a7bcc-2f58-4716-a4d0-f2a6f336ab15 LogTimestamp=2021-05-28T00:51:58.275253Z Entering 2021-05-28T00:51:58.275Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c3cc8a40-334a-40f8-a2ac-08e40b13a11b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:51:58.275215Z InvocationID=989a7bcc-2f58-4716-a4d0-f2a6f336ab15 LogTimestamp=2021-05-28T00:51:58.275253Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:51:58.355Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:51:58.275Z InvocationID=b4909a5b-6957-429f-9bbf-825b2b0ce9e9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:51:58.360Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:51:58.275Z InvocationID=b4909a5b-6957-429f-9bbf-825b2b0ce9e9 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:51:58.361Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:51:58.275Z InvocationID=b4909a5b-6957-429f-9bbf-825b2b0ce9e9 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:51:58.361Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=86 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:51:58.275Z InvocationID=b4909a5b-6957-429f-9bbf-825b2b0ce9e9 ErrorCode=500 LogTimestamp=2021-05-28T00:51:58.361678Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:51:58.362Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:52:00.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e1bef5e-01b2-44fc-8cb4-7a996d257ed1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:00.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e1bef5e-01b2-44fc-8cb4-7a996d257ed1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5073e7e0 2021-05-28T00:52:00.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e1bef5e-01b2-44fc-8cb4-7a996d257ed1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:00.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e1bef5e-01b2-44fc-8cb4-7a996d257ed1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:02.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=661713f5-ea3c-4d8e-b7a3-522737e90cda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:02.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=661713f5-ea3c-4d8e-b7a3-522737e90cda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5315f85e 2021-05-28T00:52:02.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=661713f5-ea3c-4d8e-b7a3-522737e90cda PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:02.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=661713f5-ea3c-4d8e-b7a3-522737e90cda PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:05.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7098def3-6614-4af1-b4e8-58c2ac1fdb28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:05.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7098def3-6614-4af1-b4e8-58c2ac1fdb28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@282235c 2021-05-28T00:52:05.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7098def3-6614-4af1-b4e8-58c2ac1fdb28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:05.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7098def3-6614-4af1-b4e8-58c2ac1fdb28 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:52:05.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:52:08.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16fea2c6-0332-449c-8f18-6bf98f73a74b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:08.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16fea2c6-0332-449c-8f18-6bf98f73a74b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a3f1c8f 2021-05-28T00:52:08.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16fea2c6-0332-449c-8f18-6bf98f73a74b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:08.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16fea2c6-0332-449c-8f18-6bf98f73a74b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:08.761Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1d1c4a61-ff36-45de-be4a-9cca81d4f2d6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:08.761263Z InvocationID=d376c7c3-3a3b-47f7-b352-6ef2af93d538 LogTimestamp=2021-05-28T00:52:08.761316Z Entering 2021-05-28T00:52:08.761Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1d1c4a61-ff36-45de-be4a-9cca81d4f2d6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:08.761263Z InvocationID=d376c7c3-3a3b-47f7-b352-6ef2af93d538 LogTimestamp=2021-05-28T00:52:08.761316Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:52:08.866Z [qtp580718781-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:52:08.762Z InvocationID=ce8a2705-3625-4273-9f9e-e90733e2beef PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:52:08.872Z [qtp580718781-17] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:08.762Z InvocationID=ce8a2705-3625-4273-9f9e-e90733e2beef ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:52:08.872Z [qtp580718781-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:08.762Z InvocationID=ce8a2705-3625-4273-9f9e-e90733e2beef ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:52:08.873Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=111 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:52:08.762Z InvocationID=ce8a2705-3625-4273-9f9e-e90733e2beef ErrorCode=500 LogTimestamp=2021-05-28T00:52:08.873431Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:52:08.873Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:52:11.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b8f78e0-958d-4de9-8a19-42c5c3493e79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:11.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b8f78e0-958d-4de9-8a19-42c5c3493e79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@553ca4cf 2021-05-28T00:52:11.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b8f78e0-958d-4de9-8a19-42c5c3493e79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:11.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b8f78e0-958d-4de9-8a19-42c5c3493e79 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a40345-eb02-4e37-a828-a5f4ac77480f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a40345-eb02-4e37-a828-a5f4ac77480f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2917613e 2021-05-28T00:52:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a40345-eb02-4e37-a828-a5f4ac77480f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:13.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3a40345-eb02-4e37-a828-a5f4ac77480f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:17.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6facfa6-2b5e-465e-9762-4336ae805e7f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:17.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6facfa6-2b5e-465e-9762-4336ae805e7f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a9c3f83 2021-05-28T00:52:17.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6facfa6-2b5e-465e-9762-4336ae805e7f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:17.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6facfa6-2b5e-465e-9762-4336ae805e7f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:18.975Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=275b92e1-9267-4360-a242-a8095d87227c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:18.974648Z InvocationID=71b49468-d9eb-490e-b947-2301f4f1436c LogTimestamp=2021-05-28T00:52:18.974721Z Entering 2021-05-28T00:52:18.975Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=275b92e1-9267-4360-a242-a8095d87227c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:18.974648Z InvocationID=71b49468-d9eb-490e-b947-2301f4f1436c LogTimestamp=2021-05-28T00:52:18.974721Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:52:19.065Z [qtp580718781-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:52:18.975Z InvocationID=ffcc27af-ee9d-4a13-a830-d206c006041a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:52:20.172Z [qtp580718781-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:18.975Z InvocationID=ffcc27af-ee9d-4a13-a830-d206c006041a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:52:20.172Z [qtp580718781-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:18.975Z InvocationID=ffcc27af-ee9d-4a13-a830-d206c006041a ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:52:20.173Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=1198 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:52:18.975Z InvocationID=ffcc27af-ee9d-4a13-a830-d206c006041a ErrorCode=500 LogTimestamp=2021-05-28T00:52:20.173263Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:52:20.173Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:52:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:52:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:52:21.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b080173-58c2-4963-a1aa-4f04eb9dedf2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:21.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b080173-58c2-4963-a1aa-4f04eb9dedf2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5825873b 2021-05-28T00:52:21.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b080173-58c2-4963-a1aa-4f04eb9dedf2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:21.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b080173-58c2-4963-a1aa-4f04eb9dedf2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:23.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=355ccf4d-1197-4b55-b008-e938f9270544 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:23.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=355ccf4d-1197-4b55-b008-e938f9270544 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@680f4a2c 2021-05-28T00:52:23.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=355ccf4d-1197-4b55-b008-e938f9270544 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:23.177Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=355ccf4d-1197-4b55-b008-e938f9270544 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:26.183Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=355ccf4d-1197-4b55-b008-e938f9270544 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:52:26.183Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=355ccf4d-1197-4b55-b008-e938f9270544 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:52:26.183Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=355ccf4d-1197-4b55-b008-e938f9270544 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:52:26.183Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=355ccf4d-1197-4b55-b008-e938f9270544 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:52:31.260Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:52:31.267Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T00:52:32.765Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0908b83d-c903-44d6-8640-dda519b5bed6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:32.765289Z InvocationID=961681f3-b26c-43eb-8069-17cfd43dd185 LogTimestamp=2021-05-28T00:52:32.765387Z Entering 2021-05-28T00:52:32.765Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0908b83d-c903-44d6-8640-dda519b5bed6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:32.765289Z InvocationID=961681f3-b26c-43eb-8069-17cfd43dd185 LogTimestamp=2021-05-28T00:52:32.765387Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:52:32.858Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:52:32.765Z InvocationID=12ac985d-ead8-4f39-8059-3ee08e81c041 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:52:33.168Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:32.765Z InvocationID=12ac985d-ead8-4f39-8059-3ee08e81c041 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:52:33.169Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:32.765Z InvocationID=12ac985d-ead8-4f39-8059-3ee08e81c041 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:52:33.169Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:52:32.765Z InvocationID=12ac985d-ead8-4f39-8059-3ee08e81c041 ErrorCode=500 LogTimestamp=2021-05-28T00:52:33.169621Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:52:33.170Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:52:35.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343e9dce-eb82-4ff2-8a57-38c7248fd5ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:35.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343e9dce-eb82-4ff2-8a57-38c7248fd5ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a83c882 2021-05-28T00:52:35.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343e9dce-eb82-4ff2-8a57-38c7248fd5ce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:35.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343e9dce-eb82-4ff2-8a57-38c7248fd5ce PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:52:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:52:35.660Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=343e9dce-eb82-4ff2-8a57-38c7248fd5ce ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:52:35.661Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=343e9dce-eb82-4ff2-8a57-38c7248fd5ce ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:52:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:52:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:52:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:52:38.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c11b6b6d-7966-4bc7-b19c-9a2bed76dada PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:38.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c11b6b6d-7966-4bc7-b19c-9a2bed76dada PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5550d735 2021-05-28T00:52:38.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c11b6b6d-7966-4bc7-b19c-9a2bed76dada PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:38.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c11b6b6d-7966-4bc7-b19c-9a2bed76dada PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:39.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9572f698-48a7-4c14-aa9f-af10417fb693 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:39.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9572f698-48a7-4c14-aa9f-af10417fb693 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@471ca999 2021-05-28T00:52:39.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9572f698-48a7-4c14-aa9f-af10417fb693 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:39.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9572f698-48a7-4c14-aa9f-af10417fb693 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:52:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:52:42.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b538722-ed13-43df-a979-c7f5a4681633 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:42.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b538722-ed13-43df-a979-c7f5a4681633 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@233fbe20 2021-05-28T00:52:42.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b538722-ed13-43df-a979-c7f5a4681633 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:42.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b538722-ed13-43df-a979-c7f5a4681633 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:43.574Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f50d2e5b-897f-47a5-bb28-0ab99d458d46 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:43.574348Z InvocationID=f7cc962f-ed83-4d99-ac22-cd39191e1ee8 LogTimestamp=2021-05-28T00:52:43.574586Z Entering 2021-05-28T00:52:43.575Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f50d2e5b-897f-47a5-bb28-0ab99d458d46 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:43.574348Z InvocationID=f7cc962f-ed83-4d99-ac22-cd39191e1ee8 LogTimestamp=2021-05-28T00:52:43.574586Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:52:43.575Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:43.575Z InvocationID=7ac0f9d4-90c7-4f9c-9d68-84ae850258ea ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T00:52:43.576Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:52:43.575Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:52:43.575Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=7ac0f9d4-90c7-4f9c-9d68-84ae850258ea ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5c3c9620 2021-05-28T00:52:43.576Z [qtp580718781-11] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=0 EndTimestamp=2021-05-28T00:52:43.575Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.181 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T00:52:43.575Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=7ac0f9d4-90c7-4f9c-9d68-84ae850258ea ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5c3c9620 2021-05-28T00:52:43.576Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:52:43.576339Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:52:43.576Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:52:43.576339Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:52:43.576Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:52:43.672Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:52:45.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cab1f78-0abd-4ec5-8d2c-eb03de330950 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:45.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cab1f78-0abd-4ec5-8d2c-eb03de330950 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46613111 2021-05-28T00:52:45.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cab1f78-0abd-4ec5-8d2c-eb03de330950 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:45.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cab1f78-0abd-4ec5-8d2c-eb03de330950 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:46.367Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7cab1f78-0abd-4ec5-8d2c-eb03de330950 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:52:46.367Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=7cab1f78-0abd-4ec5-8d2c-eb03de330950 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:52:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@155baf97 2021-05-28T00:52:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:52:50.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:52:52.112Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:52:53.065Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:52:53.066Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:52:53.066Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:52:53.066Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f4557ffe-5238-4c78-bf03-1774b54f0e04 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:52:53.770Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5f7d5033-5606-49b2-96a8-b61f1f6649d3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:53.770406Z InvocationID=c50982c1-ecea-4eb6-ac6a-1671ab58d5bb LogTimestamp=2021-05-28T00:52:53.770479Z Entering 2021-05-28T00:52:53.770Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5f7d5033-5606-49b2-96a8-b61f1f6649d3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:52:53.770406Z InvocationID=c50982c1-ecea-4eb6-ac6a-1671ab58d5bb LogTimestamp=2021-05-28T00:52:53.770479Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:52:53.869Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:52:53.771Z InvocationID=dfde52c7-bd2e-4456-93d7-a08d4469540d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:52:54.068Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:53.771Z InvocationID=dfde52c7-bd2e-4456-93d7-a08d4469540d ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:52:54.068Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:52:53.771Z InvocationID=dfde52c7-bd2e-4456-93d7-a08d4469540d ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:52:54.069Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=298 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:52:53.771Z InvocationID=dfde52c7-bd2e-4456-93d7-a08d4469540d ErrorCode=500 LogTimestamp=2021-05-28T00:52:54.069149Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:52:54.069Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:52:54.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ed5cc0-b885-4453-b2c1-0777b9cc4731 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:52:54.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ed5cc0-b885-4453-b2c1-0777b9cc4731 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37a98cc4 2021-05-28T00:52:54.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ed5cc0-b885-4453-b2c1-0777b9cc4731 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:52:54.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95ed5cc0-b885-4453-b2c1-0777b9cc4731 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:52:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:52:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:52:57.404Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=95ed5cc0-b885-4453-b2c1-0777b9cc4731 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:52:57.404Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=95ed5cc0-b885-4453-b2c1-0777b9cc4731 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:52:58.978Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:53:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:53:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:53:13.871Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3e283dc5-2540-4d19-8ac3-a8844a18792e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:53:13.871294Z InvocationID=693a8396-49e2-474c-9e43-cd4a20340e96 LogTimestamp=2021-05-28T00:53:13.871374Z Entering 2021-05-28T00:53:13.871Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3e283dc5-2540-4d19-8ac3-a8844a18792e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:53:13.871294Z InvocationID=693a8396-49e2-474c-9e43-cd4a20340e96 LogTimestamp=2021-05-28T00:53:13.871374Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:53:13.966Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:53:13.871Z InvocationID=e681d1c8-72e3-4aeb-a275-43271d7a660f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:53:14.456Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:53:13.871Z InvocationID=e681d1c8-72e3-4aeb-a275-43271d7a660f ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:53:14.456Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:53:13.871Z InvocationID=e681d1c8-72e3-4aeb-a275-43271d7a660f ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:53:14.457Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=586 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:53:13.871Z InvocationID=e681d1c8-72e3-4aeb-a275-43271d7a660f ErrorCode=500 LogTimestamp=2021-05-28T00:53:14.457229Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:53:14.457Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:53:17.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebc5eb5f-a82f-4a2a-941c-52df256ffa8a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:17.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebc5eb5f-a82f-4a2a-941c-52df256ffa8a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@269fa720 2021-05-28T00:53:17.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebc5eb5f-a82f-4a2a-941c-52df256ffa8a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:17.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebc5eb5f-a82f-4a2a-941c-52df256ffa8a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:17.279Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ebc5eb5f-a82f-4a2a-941c-52df256ffa8a ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:53:17.279Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=ebc5eb5f-a82f-4a2a-941c-52df256ffa8a ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:53:17.976Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e43f6e3-b480-46d6-8433-9e310aed9a85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:17.976Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e43f6e3-b480-46d6-8433-9e310aed9a85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1544fdc5 2021-05-28T00:53:17.976Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e43f6e3-b480-46d6-8433-9e310aed9a85 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:17.976Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e43f6e3-b480-46d6-8433-9e310aed9a85 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:18.278Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4e43f6e3-b480-46d6-8433-9e310aed9a85 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:53:18.278Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4e43f6e3-b480-46d6-8433-9e310aed9a85 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:53:19.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a710e62-9fb6-404b-bdc0-a3d9d0d707b6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:19.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a710e62-9fb6-404b-bdc0-a3d9d0d707b6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@330e9140 2021-05-28T00:53:19.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a710e62-9fb6-404b-bdc0-a3d9d0d707b6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:19.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a710e62-9fb6-404b-bdc0-a3d9d0d707b6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:20.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93a09881-9b44-49ed-b4b4-a835cfa907f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:20.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93a09881-9b44-49ed-b4b4-a835cfa907f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@687386a6 2021-05-28T00:53:20.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93a09881-9b44-49ed-b4b4-a835cfa907f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:20.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93a09881-9b44-49ed-b4b4-a835cfa907f3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:53:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:53:21.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2907f5-36d6-41dd-a15e-c78cfb320295 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:21.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2907f5-36d6-41dd-a15e-c78cfb320295 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f2ebd0 2021-05-28T00:53:21.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2907f5-36d6-41dd-a15e-c78cfb320295 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:21.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2907f5-36d6-41dd-a15e-c78cfb320295 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:23.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32785a99-8d18-4c6f-a54a-e9c977dde0ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:23.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32785a99-8d18-4c6f-a54a-e9c977dde0ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@306a849e 2021-05-28T00:53:23.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32785a99-8d18-4c6f-a54a-e9c977dde0ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:23.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32785a99-8d18-4c6f-a54a-e9c977dde0ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:25.072Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b34dbc19-f971-4bd3-bcbb-249ec3921256 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:53:25.071683Z InvocationID=a512094a-e916-44eb-ad32-c51167271dc8 LogTimestamp=2021-05-28T00:53:25.071773Z Entering 2021-05-28T00:53:25.072Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b34dbc19-f971-4bd3-bcbb-249ec3921256 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.181 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.181 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:53:25.071683Z InvocationID=a512094a-e916-44eb-ad32-c51167271dc8 LogTimestamp=2021-05-28T00:53:25.071773Z localAddr=10.242.102.221 remoteAddr=10.242.170.181 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:53:25.165Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:53:25.072Z InvocationID=a165e479-1190-435b-ad2e-48710372f04f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:53:25.263Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:53:25.072Z InvocationID=a165e479-1190-435b-ad2e-48710372f04f ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:53:25.263Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:53:25.072Z InvocationID=a165e479-1190-435b-ad2e-48710372f04f ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:53:25.264Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:53:25.072Z InvocationID=a165e479-1190-435b-ad2e-48710372f04f ErrorCode=500 LogTimestamp=2021-05-28T00:53:25.264025Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:53:25.264Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:53:26.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7825cda1-3b51-4e28-a319-3606b2b42c45 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:26.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7825cda1-3b51-4e28-a319-3606b2b42c45 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77298742 2021-05-28T00:53:26.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7825cda1-3b51-4e28-a319-3606b2b42c45 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:26.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7825cda1-3b51-4e28-a319-3606b2b42c45 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:29.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37db4cf-acca-473d-b168-5481f9d19740 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:29.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37db4cf-acca-473d-b168-5481f9d19740 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39b79b7 2021-05-28T00:53:29.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37db4cf-acca-473d-b168-5481f9d19740 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:29.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f37db4cf-acca-473d-b168-5481f9d19740 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:31.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f72a7ef1-0a22-4e86-8f00-0751aac3ff27 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:31.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f72a7ef1-0a22-4e86-8f00-0751aac3ff27 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1622ae7f 2021-05-28T00:53:31.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f72a7ef1-0a22-4e86-8f00-0751aac3ff27 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:31.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f72a7ef1-0a22-4e86-8f00-0751aac3ff27 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:33.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=455cc8a5-3f78-4c31-950c-63fd14047926 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:33.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=455cc8a5-3f78-4c31-950c-63fd14047926 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72888651 2021-05-28T00:53:33.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=455cc8a5-3f78-4c31-950c-63fd14047926 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:33.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=455cc8a5-3f78-4c31-950c-63fd14047926 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:53:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:53:36.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=865374ce-3e7e-44c2-973a-77acfbad7cae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:36.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=865374ce-3e7e-44c2-973a-77acfbad7cae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49a00cd5 2021-05-28T00:53:36.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=865374ce-3e7e-44c2-973a-77acfbad7cae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:36.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=865374ce-3e7e-44c2-973a-77acfbad7cae PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:53:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:53:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:53:37.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b84dd6e-1b41-4749-be98-0b93061608be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:37.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b84dd6e-1b41-4749-be98-0b93061608be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11ae9297 2021-05-28T00:53:37.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b84dd6e-1b41-4749-be98-0b93061608be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:37.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b84dd6e-1b41-4749-be98-0b93061608be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:39.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b50b30b-dedf-406e-bbe8-55308129b2d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:39.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b50b30b-dedf-406e-bbe8-55308129b2d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@127c978b 2021-05-28T00:53:39.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b50b30b-dedf-406e-bbe8-55308129b2d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:39.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b50b30b-dedf-406e-bbe8-55308129b2d9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:40.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=665e2e16-90c4-426a-add7-b18eed3e15fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:40.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=665e2e16-90c4-426a-add7-b18eed3e15fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24f0319b 2021-05-28T00:53:40.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=665e2e16-90c4-426a-add7-b18eed3e15fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:40.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=665e2e16-90c4-426a-add7-b18eed3e15fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:41.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bea475d7-5322-4b3a-8c65-1f88314d1cb0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:41.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bea475d7-5322-4b3a-8c65-1f88314d1cb0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d7f4774 2021-05-28T00:53:41.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bea475d7-5322-4b3a-8c65-1f88314d1cb0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:41.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bea475d7-5322-4b3a-8c65-1f88314d1cb0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:42.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e46d6e-33d1-4f5e-8e3b-00a6bc97968a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:42.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e46d6e-33d1-4f5e-8e3b-00a6bc97968a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8204a66 2021-05-28T00:53:42.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e46d6e-33d1-4f5e-8e3b-00a6bc97968a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:42.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e46d6e-33d1-4f5e-8e3b-00a6bc97968a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:44.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f9cbb7-8cca-4af8-8749-4136a02652be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:44.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f9cbb7-8cca-4af8-8749-4136a02652be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e965426 2021-05-28T00:53:44.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f9cbb7-8cca-4af8-8749-4136a02652be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:44.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f9cbb7-8cca-4af8-8749-4136a02652be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:45.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9317c034-d31b-4fee-a320-aedce38cbdce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:45.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9317c034-d31b-4fee-a320-aedce38cbdce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5bcfde86 2021-05-28T00:53:45.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9317c034-d31b-4fee-a320-aedce38cbdce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:45.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9317c034-d31b-4fee-a320-aedce38cbdce PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:46.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52996649-f05b-43df-ae54-0c5c899d6240 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:46.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52996649-f05b-43df-ae54-0c5c899d6240 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a1e2d6d 2021-05-28T00:53:46.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52996649-f05b-43df-ae54-0c5c899d6240 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:46.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52996649-f05b-43df-ae54-0c5c899d6240 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:47.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53eb391-2085-4104-95ce-f7d7c2e1a945 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:47.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53eb391-2085-4104-95ce-f7d7c2e1a945 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f80f946 2021-05-28T00:53:47.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53eb391-2085-4104-95ce-f7d7c2e1a945 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:47.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53eb391-2085-4104-95ce-f7d7c2e1a945 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:48.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c22a19a7-a650-45c3-88f9-30477e08103a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:48.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c22a19a7-a650-45c3-88f9-30477e08103a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45dc 2021-05-28T00:53:48.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c22a19a7-a650-45c3-88f9-30477e08103a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:48.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c22a19a7-a650-45c3-88f9-30477e08103a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:49.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2975560f-21aa-4392-aebf-8f8e136fe74f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:49.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2975560f-21aa-4392-aebf-8f8e136fe74f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6dbfe951 2021-05-28T00:53:49.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2975560f-21aa-4392-aebf-8f8e136fe74f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:49.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2975560f-21aa-4392-aebf-8f8e136fe74f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:53:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:53:52.201Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:53:52.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab4c4d1-8d82-40ef-b6dc-f027fc8c083f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:52.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab4c4d1-8d82-40ef-b6dc-f027fc8c083f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1631c4ce 2021-05-28T00:53:52.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab4c4d1-8d82-40ef-b6dc-f027fc8c083f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:52.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab4c4d1-8d82-40ef-b6dc-f027fc8c083f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:55.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21416a00-ec56-46f9-b1b5-6d7bb8c7336d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:55.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21416a00-ec56-46f9-b1b5-6d7bb8c7336d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ed34587 2021-05-28T00:53:55.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21416a00-ec56-46f9-b1b5-6d7bb8c7336d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:55.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21416a00-ec56-46f9-b1b5-6d7bb8c7336d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:53:58.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bcb0d7c-f3c4-435c-908d-5a724acbf74b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:53:58.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bcb0d7c-f3c4-435c-908d-5a724acbf74b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c7fd924 2021-05-28T00:53:58.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bcb0d7c-f3c4-435c-908d-5a724acbf74b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:53:58.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bcb0d7c-f3c4-435c-908d-5a724acbf74b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:01.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6997a18-ec3b-45b1-bda9-8558ced9b922 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:01.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6997a18-ec3b-45b1-bda9-8558ced9b922 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e984a9b 2021-05-28T00:54:01.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6997a18-ec3b-45b1-bda9-8558ced9b922 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:01.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6997a18-ec3b-45b1-bda9-8558ced9b922 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53dcdd4-e30c-4263-8d2d-c6f28460eca7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53dcdd4-e30c-4263-8d2d-c6f28460eca7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3617ec32 2021-05-28T00:54:04.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53dcdd4-e30c-4263-8d2d-c6f28460eca7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:04.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a53dcdd4-e30c-4263-8d2d-c6f28460eca7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:54:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:54:07.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df546ad8-a579-49f7-a001-650aacbd0477 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:07.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df546ad8-a579-49f7-a001-650aacbd0477 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12cc780a 2021-05-28T00:54:07.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df546ad8-a579-49f7-a001-650aacbd0477 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:07.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df546ad8-a579-49f7-a001-650aacbd0477 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed6e8d3-1474-42be-a37d-1bf26f7c6138 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:10.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed6e8d3-1474-42be-a37d-1bf26f7c6138 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50b8258e 2021-05-28T00:54:10.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed6e8d3-1474-42be-a37d-1bf26f7c6138 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:10.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ed6e8d3-1474-42be-a37d-1bf26f7c6138 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:13.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=529175f7-c74a-450d-bac9-be923500660c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:13.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=529175f7-c74a-450d-bac9-be923500660c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c987190 2021-05-28T00:54:13.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=529175f7-c74a-450d-bac9-be923500660c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:13.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=529175f7-c74a-450d-bac9-be923500660c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:16.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3738c2f5-a93d-4134-b9d4-4f6414ed56f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:16.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3738c2f5-a93d-4134-b9d4-4f6414ed56f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35e76c4d 2021-05-28T00:54:16.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3738c2f5-a93d-4134-b9d4-4f6414ed56f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:16.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3738c2f5-a93d-4134-b9d4-4f6414ed56f9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:19.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09852170-b22d-4ccd-9c44-ec1eca4e612b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:19.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09852170-b22d-4ccd-9c44-ec1eca4e612b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@441a5ed3 2021-05-28T00:54:19.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09852170-b22d-4ccd-9c44-ec1eca4e612b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:19.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09852170-b22d-4ccd-9c44-ec1eca4e612b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:54:20.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:54:22.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2f4c93-d9ed-4588-b397-f463180e4860 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:22.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2f4c93-d9ed-4588-b397-f463180e4860 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@551f978b 2021-05-28T00:54:22.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2f4c93-d9ed-4588-b397-f463180e4860 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:22.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2f4c93-d9ed-4588-b397-f463180e4860 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:25.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d472fa3-a27a-4b05-b354-2ff6842773f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:25.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d472fa3-a27a-4b05-b354-2ff6842773f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a746849 2021-05-28T00:54:25.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d472fa3-a27a-4b05-b354-2ff6842773f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:25.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d472fa3-a27a-4b05-b354-2ff6842773f2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:28.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7262d0a7-e76a-4073-ab4d-9e18f9a14cd1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:28.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7262d0a7-e76a-4073-ab4d-9e18f9a14cd1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@247ffc4e 2021-05-28T00:54:28.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7262d0a7-e76a-4073-ab4d-9e18f9a14cd1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:28.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7262d0a7-e76a-4073-ab4d-9e18f9a14cd1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:31.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47b55ede-d3c8-4552-9c29-b276e2d92026 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:31.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47b55ede-d3c8-4552-9c29-b276e2d92026 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3205d2d4 2021-05-28T00:54:31.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47b55ede-d3c8-4552-9c29-b276e2d92026 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:31.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47b55ede-d3c8-4552-9c29-b276e2d92026 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:35.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da1f0743-c83d-4e2d-a1d8-376b32be673c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:35.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da1f0743-c83d-4e2d-a1d8-376b32be673c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b8fa8d0 2021-05-28T00:54:35.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da1f0743-c83d-4e2d-a1d8-376b32be673c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:35.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da1f0743-c83d-4e2d-a1d8-376b32be673c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:54:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:54:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:54:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:54:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:54:37.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab1ddd4-c714-42db-90e1-ae3d098a9080 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:37.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab1ddd4-c714-42db-90e1-ae3d098a9080 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12b0adc8 2021-05-28T00:54:37.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab1ddd4-c714-42db-90e1-ae3d098a9080 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:37.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab1ddd4-c714-42db-90e1-ae3d098a9080 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:41.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71834e9e-4c68-4e7e-9549-f6a0a50abf28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:41.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71834e9e-4c68-4e7e-9549-f6a0a50abf28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49f6ae5c 2021-05-28T00:54:41.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71834e9e-4c68-4e7e-9549-f6a0a50abf28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:41.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71834e9e-4c68-4e7e-9549-f6a0a50abf28 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:44.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc4cf31d-caed-411c-bcb6-ac433878c008 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:44.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc4cf31d-caed-411c-bcb6-ac433878c008 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55f946c4 2021-05-28T00:54:44.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc4cf31d-caed-411c-bcb6-ac433878c008 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:44.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc4cf31d-caed-411c-bcb6-ac433878c008 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:46.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a48e77-0924-46fa-bfcf-b40c8db735fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:46.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a48e77-0924-46fa-bfcf-b40c8db735fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33d6bd83 2021-05-28T00:54:46.982Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a48e77-0924-46fa-bfcf-b40c8db735fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:46.983Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a48e77-0924-46fa-bfcf-b40c8db735fd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:50.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8d4ee02-e645-48d8-a779-d27b4c237400 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:50.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8d4ee02-e645-48d8-a779-d27b4c237400 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@241f88dc 2021-05-28T00:54:50.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8d4ee02-e645-48d8-a779-d27b4c237400 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:50.086Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8d4ee02-e645-48d8-a779-d27b4c237400 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:54:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:54:52.125Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:54:53.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2a999b-f91e-4248-9696-5e9a297851b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:53.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2a999b-f91e-4248-9696-5e9a297851b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5867464d 2021-05-28T00:54:53.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2a999b-f91e-4248-9696-5e9a297851b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:53.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b2a999b-f91e-4248-9696-5e9a297851b4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:55.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf278ac1-875f-4150-8afe-adec3385b07b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:55.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf278ac1-875f-4150-8afe-adec3385b07b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47b7f4e3 2021-05-28T00:54:55.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf278ac1-875f-4150-8afe-adec3385b07b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:55.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf278ac1-875f-4150-8afe-adec3385b07b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:54:59.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf2322f-dba2-4d38-880b-db0e4fab9277 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:54:59.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf2322f-dba2-4d38-880b-db0e4fab9277 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@410b7333 2021-05-28T00:54:59.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf2322f-dba2-4d38-880b-db0e4fab9277 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:54:59.377Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf2322f-dba2-4d38-880b-db0e4fab9277 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:01.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a20c5a28-0a2f-493c-9c44-11309013222a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:01.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a20c5a28-0a2f-493c-9c44-11309013222a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@198a4aa6 2021-05-28T00:55:01.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a20c5a28-0a2f-493c-9c44-11309013222a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:01.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a20c5a28-0a2f-493c-9c44-11309013222a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:04.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54c4aa3b-02c2-4191-a67f-9f2a35ff9798 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:04.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54c4aa3b-02c2-4191-a67f-9f2a35ff9798 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b3d22f4 2021-05-28T00:55:04.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54c4aa3b-02c2-4191-a67f-9f2a35ff9798 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:04.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54c4aa3b-02c2-4191-a67f-9f2a35ff9798 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:55:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:55:08.260Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a757eae4-3ae6-423c-893c-92cb95d0535c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:08.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a757eae4-3ae6-423c-893c-92cb95d0535c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@355d19c 2021-05-28T00:55:08.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a757eae4-3ae6-423c-893c-92cb95d0535c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:08.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a757eae4-3ae6-423c-893c-92cb95d0535c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:11.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b063474-84b6-4513-98d1-3b1e1978676f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:11.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b063474-84b6-4513-98d1-3b1e1978676f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19dda82e 2021-05-28T00:55:11.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b063474-84b6-4513-98d1-3b1e1978676f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:11.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b063474-84b6-4513-98d1-3b1e1978676f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9c3e59-2f34-4395-bfbb-f1c2c4ebea70 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:13.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9c3e59-2f34-4395-bfbb-f1c2c4ebea70 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52c90ec9 2021-05-28T00:55:13.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9c3e59-2f34-4395-bfbb-f1c2c4ebea70 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:13.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9c3e59-2f34-4395-bfbb-f1c2c4ebea70 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:16.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3cb228b1-8537-40a5-86a4-eabb06dadaba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3cb228b1-8537-40a5-86a4-eabb06dadaba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4071bce6 2021-05-28T00:55:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3cb228b1-8537-40a5-86a4-eabb06dadaba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3cb228b1-8537-40a5-86a4-eabb06dadaba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:20.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913887d-3e0e-4be6-b565-f6402e260f0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:20.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913887d-3e0e-4be6-b565-f6402e260f0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d055c45 2021-05-28T00:55:20.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913887d-3e0e-4be6-b565-f6402e260f0d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:20.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5913887d-3e0e-4be6-b565-f6402e260f0d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:55:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:55:23.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=752bd0bc-ce3e-4478-bc05-28fc071ac1b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:23.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=752bd0bc-ce3e-4478-bc05-28fc071ac1b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@196d0513 2021-05-28T00:55:23.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=752bd0bc-ce3e-4478-bc05-28fc071ac1b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:23.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=752bd0bc-ce3e-4478-bc05-28fc071ac1b8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:26.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e426201c-0a10-4869-96f0-27726664db59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:26.482Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e426201c-0a10-4869-96f0-27726664db59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@348ffb1 2021-05-28T00:55:26.483Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e426201c-0a10-4869-96f0-27726664db59 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:26.483Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e426201c-0a10-4869-96f0-27726664db59 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:29.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b071f406-fab9-4f3f-82f1-fb94f24d8aa1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:29.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b071f406-fab9-4f3f-82f1-fb94f24d8aa1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e13f5b2 2021-05-28T00:55:29.895Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b071f406-fab9-4f3f-82f1-fb94f24d8aa1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:29.895Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b071f406-fab9-4f3f-82f1-fb94f24d8aa1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:34.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8a623a0-cbe0-4e7d-9743-2c58a0e438cb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:34.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8a623a0-cbe0-4e7d-9743-2c58a0e438cb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9336f41 2021-05-28T00:55:34.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8a623a0-cbe0-4e7d-9743-2c58a0e438cb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:34.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8a623a0-cbe0-4e7d-9743-2c58a0e438cb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:35.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89ca28b4-71ba-4e40-a9cb-82ea0752e063 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:35.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89ca28b4-71ba-4e40-a9cb-82ea0752e063 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7807a6e9 2021-05-28T00:55:35.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89ca28b4-71ba-4e40-a9cb-82ea0752e063 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:35.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89ca28b4-71ba-4e40-a9cb-82ea0752e063 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:55:35.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:55:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:55:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:55:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:55:38.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63fba7-0411-48cf-a824-995010233275 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:38.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63fba7-0411-48cf-a824-995010233275 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c767b2f 2021-05-28T00:55:38.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63fba7-0411-48cf-a824-995010233275 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:38.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63fba7-0411-48cf-a824-995010233275 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:41.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8184f5e2-9f85-481b-a84a-8b3f142eb705 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:41.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8184f5e2-9f85-481b-a84a-8b3f142eb705 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d19a776 2021-05-28T00:55:41.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8184f5e2-9f85-481b-a84a-8b3f142eb705 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:41.379Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8184f5e2-9f85-481b-a84a-8b3f142eb705 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:44.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b7ebb3-091d-4e94-a012-97fb262086e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:44.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b7ebb3-091d-4e94-a012-97fb262086e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f4294f3 2021-05-28T00:55:44.889Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b7ebb3-091d-4e94-a012-97fb262086e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:44.890Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b7ebb3-091d-4e94-a012-97fb262086e1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:46.921Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d633db4e-edc4-4549-83ef-8b349e83128d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:46.921Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d633db4e-edc4-4549-83ef-8b349e83128d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ab7551c 2021-05-28T00:55:46.921Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d633db4e-edc4-4549-83ef-8b349e83128d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:46.921Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d633db4e-edc4-4549-83ef-8b349e83128d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:50.284Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=456b4d3c-072a-44d8-bb89-a2cef5711287 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:50.284Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=456b4d3c-072a-44d8-bb89-a2cef5711287 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3597fe66 2021-05-28T00:55:50.284Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=456b4d3c-072a-44d8-bb89-a2cef5711287 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:50.284Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=456b4d3c-072a-44d8-bb89-a2cef5711287 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:55:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:55:52.118Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:55:53.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14fb8964-1497-4b93-8ff6-85b509035588 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:53.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14fb8964-1497-4b93-8ff6-85b509035588 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1595ddb5 2021-05-28T00:55:53.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14fb8964-1497-4b93-8ff6-85b509035588 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:53.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14fb8964-1497-4b93-8ff6-85b509035588 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:55.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd7e44e-01f6-415b-8e1d-302a878467b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:55.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd7e44e-01f6-415b-8e1d-302a878467b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e9b26f8 2021-05-28T00:55:55.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd7e44e-01f6-415b-8e1d-302a878467b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:55.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbd7e44e-01f6-415b-8e1d-302a878467b7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:55:58.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d8fc651-8c80-4803-8a96-1a0999cde361 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:55:58.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d8fc651-8c80-4803-8a96-1a0999cde361 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e048382 2021-05-28T00:55:58.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d8fc651-8c80-4803-8a96-1a0999cde361 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:55:58.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d8fc651-8c80-4803-8a96-1a0999cde361 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:01.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abd3e0cf-260a-49dc-9bed-7da95a700ad3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:01.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abd3e0cf-260a-49dc-9bed-7da95a700ad3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@746a36ea 2021-05-28T00:56:01.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abd3e0cf-260a-49dc-9bed-7da95a700ad3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:01.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abd3e0cf-260a-49dc-9bed-7da95a700ad3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:05.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35f62050-5c98-4f81-9d7a-799f2fcac1a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:05.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35f62050-5c98-4f81-9d7a-799f2fcac1a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f2925a7 2021-05-28T00:56:05.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35f62050-5c98-4f81-9d7a-799f2fcac1a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:05.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35f62050-5c98-4f81-9d7a-799f2fcac1a3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:56:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:56:08.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49421d3-c4d8-4855-a9ab-3c2e46fff82b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:08.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49421d3-c4d8-4855-a9ab-3c2e46fff82b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6822765c 2021-05-28T00:56:08.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49421d3-c4d8-4855-a9ab-3c2e46fff82b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:08.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a49421d3-c4d8-4855-a9ab-3c2e46fff82b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:10.909Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdfc9a5a-cacd-486b-b2a8-16d7d6eb1023 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:10.909Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdfc9a5a-cacd-486b-b2a8-16d7d6eb1023 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32e0f049 2021-05-28T00:56:10.909Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdfc9a5a-cacd-486b-b2a8-16d7d6eb1023 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:10.909Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdfc9a5a-cacd-486b-b2a8-16d7d6eb1023 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:13.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36980643-f7a5-4949-bb9d-adfef28adc81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:13.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36980643-f7a5-4949-bb9d-adfef28adc81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17985d99 2021-05-28T00:56:13.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36980643-f7a5-4949-bb9d-adfef28adc81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:13.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36980643-f7a5-4949-bb9d-adfef28adc81 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58690be1-cb27-4caa-911a-3891b703b125 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58690be1-cb27-4caa-911a-3891b703b125 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@111a46d0 2021-05-28T00:56:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58690be1-cb27-4caa-911a-3891b703b125 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58690be1-cb27-4caa-911a-3891b703b125 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:19.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=292e80ef-d196-4eea-b8d2-63d46b866ed9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:19.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=292e80ef-d196-4eea-b8d2-63d46b866ed9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6af15239 2021-05-28T00:56:19.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=292e80ef-d196-4eea-b8d2-63d46b866ed9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:19.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=292e80ef-d196-4eea-b8d2-63d46b866ed9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:56:20.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:56:20.575Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=11190b80-531b-4cea-9684-cc74310a42a4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:20.574606Z InvocationID=160c452d-3e31-4f0f-a489-22aea8d63317 LogTimestamp=2021-05-28T00:56:20.574735Z Entering 2021-05-28T00:56:20.575Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=11190b80-531b-4cea-9684-cc74310a42a4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:20.574606Z InvocationID=160c452d-3e31-4f0f-a489-22aea8d63317 LogTimestamp=2021-05-28T00:56:20.574735Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:56:21.978Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1403 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:56:20.575Z InvocationID=6259586e-a8f4-4f69-af9f-a6083f487a13 LogTimestamp=2021-05-28T00:56:21.978754Z ResponseDesc=OK Exiting. 2021-05-28T00:56:21.979Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:56:22.263Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=67440 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:55:14.822Z InvocationID=63c31e9d-149b-49c2-9615-e080f8e80032 ErrorCode=500 LogTimestamp=2021-05-28T00:56:22.262918Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:22.264Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T00:56:22.272Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b1303c11-c6a7-49b5-9fb5-03ca4c3763c7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:22.272697Z InvocationID=af7ee57e-b182-46ca-91be-ba5dedddcfc5 LogTimestamp=2021-05-28T00:56:22.272737Z Entering 2021-05-28T00:56:22.272Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b1303c11-c6a7-49b5-9fb5-03ca4c3763c7 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:22.272697Z InvocationID=af7ee57e-b182-46ca-91be-ba5dedddcfc5 LogTimestamp=2021-05-28T00:56:22.272737Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:56:22.487Z [qtp580718781-96] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:56:22.273Z InvocationID=4638a35e-75be-4dfd-9764-2a0de9815e3b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c41fc1b', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 212ms 2021-05-28T00:56:22.488Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=215 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:56:22.273Z InvocationID=4638a35e-75be-4dfd-9764-2a0de9815e3b ErrorCode=409 LogTimestamp=2021-05-28T00:56:22.488145Z ResponseDesc=Conflict Exiting. 2021-05-28T00:56:22.488Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:56:22.574Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:22.574764Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:22.575Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:22.574764Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:22.575Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T00:56:22.774Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4e468484-ca5a-4da6-a118-238f4559d309 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:22.773841Z InvocationID=e1932fd9-9060-4ce6-ba1e-32989f28961d LogTimestamp=2021-05-28T00:56:22.773898Z Entering 2021-05-28T00:56:22.774Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4e468484-ca5a-4da6-a118-238f4559d309 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:22.773841Z InvocationID=e1932fd9-9060-4ce6-ba1e-32989f28961d LogTimestamp=2021-05-28T00:56:22.773898Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:56:22.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3050ee7-4e9e-434a-a6b8-1db2e7a21a6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:22.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3050ee7-4e9e-434a-a6b8-1db2e7a21a6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f269596 2021-05-28T00:56:22.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3050ee7-4e9e-434a-a6b8-1db2e7a21a6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:22.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3050ee7-4e9e-434a-a6b8-1db2e7a21a6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:23.078Z [qtp580718781-17] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:56:22.774Z InvocationID=dd82e2fb-61cd-4d51-9f33-9ad34913ec01 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c41fc1b', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 303ms 2021-05-28T00:56:23.079Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=305 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:56:22.774Z InvocationID=dd82e2fb-61cd-4d51-9f33-9ad34913ec01 ErrorCode=400 LogTimestamp=2021-05-28T00:56:23.079395Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:56:23.079Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:56:23.274Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=76616 PartnerName=curl/7.64.0 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:55:06.658Z InvocationID=a168a029-d100-4721-9b4b-8022e9a474d0 ErrorCode=500 LogTimestamp=2021-05-28T00:56:23.274002Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:23.274Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T00:56:23.477Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=44cd7d40-dad5-4364-9c30-631bc49016b0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:23.477049Z InvocationID=2bba1554-ad8a-4ca3-aafc-4800ee229856 LogTimestamp=2021-05-28T00:56:23.477092Z Entering 2021-05-28T00:56:23.477Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=44cd7d40-dad5-4364-9c30-631bc49016b0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:23.477049Z InvocationID=2bba1554-ad8a-4ca3-aafc-4800ee229856 LogTimestamp=2021-05-28T00:56:23.477092Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:56:23.769Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:56:23.477Z InvocationID=0bf7006e-60a5-4d7c-ac93-d9c05f814f0d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c41fc1b', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 290ms 2021-05-28T00:56:23.769Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=292 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T00:56:23.477Z InvocationID=0bf7006e-60a5-4d7c-ac93-d9c05f814f0d ErrorCode=400 LogTimestamp=2021-05-28T00:56:23.769619Z ResponseDesc=Bad Request Exiting. 2021-05-28T00:56:23.770Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T00:56:23.957Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:23.957201Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:23.957Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:23.957201Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:23.957Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T00:56:24.099Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5da5521b-2595-4b2e-9a1c-924106ee4372 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:24.099380Z InvocationID=3d4c14d1-f321-48e2-9b02-16e5f9a78953 LogTimestamp=2021-05-28T00:56:24.099421Z Entering 2021-05-28T00:56:24.099Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5da5521b-2595-4b2e-9a1c-924106ee4372 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:24.099380Z InvocationID=3d4c14d1-f321-48e2-9b02-16e5f9a78953 LogTimestamp=2021-05-28T00:56:24.099421Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:56:24.326Z [qtp580718781-18] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:56:24.099Z InvocationID=676b21fa-2645-4fca-930c-0ef017c663b0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c41fc1b', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 226ms 2021-05-28T00:56:24.326Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=227 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:56:24.099Z InvocationID=676b21fa-2645-4fca-930c-0ef017c663b0 ErrorCode=409 LogTimestamp=2021-05-28T00:56:24.326802Z ResponseDesc=Conflict Exiting. 2021-05-28T00:56:24.327Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:56:24.386Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:24.386775Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:24.386Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:24.386775Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:24.387Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T00:56:24.570Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=dab6e485-5929-4262-b9c1-75dd84ba7143 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:24.569796Z InvocationID=94427c1c-b096-4fb3-8f99-edcaeb928c49 LogTimestamp=2021-05-28T00:56:24.569847Z Entering 2021-05-28T00:56:24.570Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=dab6e485-5929-4262-b9c1-75dd84ba7143 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:24.569796Z InvocationID=94427c1c-b096-4fb3-8f99-edcaeb928c49 LogTimestamp=2021-05-28T00:56:24.569847Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:56:24.968Z [qtp580718781-11] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:56:24.570Z InvocationID=bb6c0198-3b36-4978-9ec2-b359225a9722 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c41fc1b', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 397ms 2021-05-28T00:56:24.969Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=399 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:56:24.570Z InvocationID=bb6c0198-3b36-4978-9ec2-b359225a9722 ErrorCode=409 LogTimestamp=2021-05-28T00:56:24.969097Z ResponseDesc=Conflict Exiting. 2021-05-28T00:56:24.969Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:56:25.161Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:25.160943Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:25.161Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:25.160943Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:25.161Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T00:56:25.270Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b718006d-9afd-44fa-8131-57f229a0eaa6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:25.270478Z InvocationID=cf541ad5-fdca-4113-a49d-8a4583f941c5 LogTimestamp=2021-05-28T00:56:25.270542Z Entering 2021-05-28T00:56:25.270Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b718006d-9afd-44fa-8131-57f229a0eaa6 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:25.270478Z InvocationID=cf541ad5-fdca-4113-a49d-8a4583f941c5 LogTimestamp=2021-05-28T00:56:25.270542Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T00:56:25.676Z [qtp580718781-98] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T00:56:25.270Z InvocationID=654b5145-f459-4ce0-aee2-6ccedcbfe038 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@2c41fc1b', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 405ms 2021-05-28T00:56:25.677Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=407 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T00:56:25.270Z InvocationID=654b5145-f459-4ce0-aee2-6ccedcbfe038 ErrorCode=409 LogTimestamp=2021-05-28T00:56:25.677730Z ResponseDesc=Conflict Exiting. 2021-05-28T00:56:25.678Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T00:56:26.060Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=272ac1a8-f09e-42b5-8843-a7fc364fe877 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:26.060651Z InvocationID=9d43d95e-1d3c-4f90-8f3c-9b4e9bcb7f8f LogTimestamp=2021-05-28T00:56:26.060716Z Entering 2021-05-28T00:56:26.061Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=272ac1a8-f09e-42b5-8843-a7fc364fe877 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:26.060651Z InvocationID=9d43d95e-1d3c-4f90-8f3c-9b4e9bcb7f8f LogTimestamp=2021-05-28T00:56:26.060716Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:56:26.160Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=99 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:56:26.061Z InvocationID=9361c725-6f80-4238-b5b8-2fb661ae94c7 LogTimestamp=2021-05-28T00:56:26.160461Z ResponseDesc=OK Exiting. 2021-05-28T00:56:26.160Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:56:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f20b58-6487-4dea-bf6e-233b903494e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f20b58-6487-4dea-bf6e-233b903494e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a877f7e 2021-05-28T00:56:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f20b58-6487-4dea-bf6e-233b903494e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:26.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85f20b58-6487-4dea-bf6e-233b903494e4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:26.257Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:26.257179Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:26.257Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:26.257179Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:26.257Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T00:56:26.269Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ba124adc-56bd-4b6e-8652-2c1e99fab697 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:26.269176Z InvocationID=2eb5bad5-62f3-4d13-8986-8448ec0cc66f LogTimestamp=2021-05-28T00:56:26.269217Z Entering 2021-05-28T00:56:26.269Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ba124adc-56bd-4b6e-8652-2c1e99fab697 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:26.269176Z InvocationID=2eb5bad5-62f3-4d13-8986-8448ec0cc66f LogTimestamp=2021-05-28T00:56:26.269217Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:26.364Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:26.269Z InvocationID=e5f5c81a-2f56-4dad-a4e2-f2d2eeeeda79 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T00:56:27.684Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1415 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:26.269Z InvocationID=e5f5c81a-2f56-4dad-a4e2-f2d2eeeeda79 LogTimestamp=2021-05-28T00:56:27.684633Z ResponseDesc=Created Exiting. 2021-05-28T00:56:27.685Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:27.775Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:27.775589Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:27.775Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:27.775589Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:27.776Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T00:56:27.787Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=07d0505b-3ae0-493d-a2b1-caf4b1395b17 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:27.787614Z InvocationID=0fd50cf3-ec16-455f-aba7-5fa2ad3b1ee2 LogTimestamp=2021-05-28T00:56:27.787667Z Entering 2021-05-28T00:56:27.788Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=07d0505b-3ae0-493d-a2b1-caf4b1395b17 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:27.787614Z InvocationID=0fd50cf3-ec16-455f-aba7-5fa2ad3b1ee2 LogTimestamp=2021-05-28T00:56:27.787667Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:27.877Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:27.788Z InvocationID=16b76b4d-bb52-4ec5-9150-2df6b83d34e1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T00:56:28.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f717448-702c-43a7-b85c-d895c6b742f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:28.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f717448-702c-43a7-b85c-d895c6b742f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49e88438 2021-05-28T00:56:28.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f717448-702c-43a7-b85c-d895c6b742f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:28.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f717448-702c-43a7-b85c-d895c6b742f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:29.476Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1688 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:27.788Z InvocationID=16b76b4d-bb52-4ec5-9150-2df6b83d34e1 LogTimestamp=2021-05-28T00:56:29.476009Z ResponseDesc=Created Exiting. 2021-05-28T00:56:29.476Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:29.571Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:29.571619Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:29.571Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:29.571619Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:29.572Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T00:56:29.583Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0c11ce79-8632-421c-ba4e-d93ecc66cc2a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:29.583240Z InvocationID=f5013a3c-48a9-42c7-9149-b16e08610383 LogTimestamp=2021-05-28T00:56:29.583308Z Entering 2021-05-28T00:56:29.583Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0c11ce79-8632-421c-ba4e-d93ecc66cc2a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:29.583240Z InvocationID=f5013a3c-48a9-42c7-9149-b16e08610383 LogTimestamp=2021-05-28T00:56:29.583308Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:29.591Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:29.583Z InvocationID=a4e47eac-bf3f-49ce-ad50-88e08ed36c07 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T00:56:31.361Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1778 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:29.583Z InvocationID=a4e47eac-bf3f-49ce-ad50-88e08ed36c07 LogTimestamp=2021-05-28T00:56:31.361693Z ResponseDesc=Created Exiting. 2021-05-28T00:56:31.362Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:31.469Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:31.469339Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:31.469Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:31.469339Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:31.469Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T00:56:31.477Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6db5076c-3829-416c-b1c6-f2de84249f55 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:31.477282Z InvocationID=c2bbb858-dbc5-4e66-be9b-b598450f78e4 LogTimestamp=2021-05-28T00:56:31.477313Z Entering 2021-05-28T00:56:31.477Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6db5076c-3829-416c-b1c6-f2de84249f55 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:31.477282Z InvocationID=c2bbb858-dbc5-4e66-be9b-b598450f78e4 LogTimestamp=2021-05-28T00:56:31.477313Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:31.570Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:31.477Z InvocationID=dadfee7a-de51-41cb-aee0-3df609dba2a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T00:56:31.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f215f-dc1b-4f10-8300-5099522bb273 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:31.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f215f-dc1b-4f10-8300-5099522bb273 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29e867a6 2021-05-28T00:56:31.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f215f-dc1b-4f10-8300-5099522bb273 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:31.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f215f-dc1b-4f10-8300-5099522bb273 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:33.367Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1890 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:31.477Z InvocationID=dadfee7a-de51-41cb-aee0-3df609dba2a1 LogTimestamp=2021-05-28T00:56:33.367347Z ResponseDesc=Created Exiting. 2021-05-28T00:56:33.367Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:33.461Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:33.460993Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:33.461Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:33.460993Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:33.461Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T00:56:33.468Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=11fd535e-1e39-47d6-93f8-9b3dd5490acd InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:33.468603Z InvocationID=f2af1bf2-a626-4e39-a7c1-e4d840b40b93 LogTimestamp=2021-05-28T00:56:33.468642Z Entering 2021-05-28T00:56:33.468Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=11fd535e-1e39-47d6-93f8-9b3dd5490acd InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:33.468603Z InvocationID=f2af1bf2-a626-4e39-a7c1-e4d840b40b93 LogTimestamp=2021-05-28T00:56:33.468642Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:33.555Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:33.468Z InvocationID=9c6b8a7b-5857-456b-8cf8-09e8c4a543f7 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T00:56:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36644826-eaa7-4f1b-809e-a6bdc5968d62 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36644826-eaa7-4f1b-809e-a6bdc5968d62 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4209dd28 2021-05-28T00:56:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36644826-eaa7-4f1b-809e-a6bdc5968d62 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36644826-eaa7-4f1b-809e-a6bdc5968d62 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:35.378Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1910 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:33.468Z InvocationID=9c6b8a7b-5857-456b-8cf8-09e8c4a543f7 LogTimestamp=2021-05-28T00:56:35.378581Z ResponseDesc=Created Exiting. 2021-05-28T00:56:35.378Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:35.469Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:35.469746Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:35.469Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:35.469746Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:35.470Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T00:56:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:56:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:56:35.564Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=41c8f3be-89b7-42df-8566-7a41a84d2c25 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:35.564545Z InvocationID=81bb04d1-b381-46ae-87da-0b4b05e83c06 LogTimestamp=2021-05-28T00:56:35.564586Z Entering 2021-05-28T00:56:35.564Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=41c8f3be-89b7-42df-8566-7a41a84d2c25 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:35.564545Z InvocationID=81bb04d1-b381-46ae-87da-0b4b05e83c06 LogTimestamp=2021-05-28T00:56:35.564586Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:35.574Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:35.565Z InvocationID=c5c6d308-df9c-4a2b-a7a3-f22115ee03fc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T00:56:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:56:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:56:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:56:37.375Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1810 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:35.565Z InvocationID=c5c6d308-df9c-4a2b-a7a3-f22115ee03fc LogTimestamp=2021-05-28T00:56:37.375279Z ResponseDesc=Created Exiting. 2021-05-28T00:56:37.375Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:37.474Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:37.474909Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:37.475Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:37.474909Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:37.475Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T00:56:37.571Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a3364c7e-3eb9-4d59-b957-6150f3916cef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:37.571328Z InvocationID=1d6a6d87-dc54-4797-8728-940beae1ee57 LogTimestamp=2021-05-28T00:56:37.571369Z Entering 2021-05-28T00:56:37.571Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a3364c7e-3eb9-4d59-b957-6150f3916cef InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:37.571328Z InvocationID=1d6a6d87-dc54-4797-8728-940beae1ee57 LogTimestamp=2021-05-28T00:56:37.571369Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:37.586Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:37.571Z InvocationID=2401156a-01b6-4109-964c-6c0d639efbdb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T00:56:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23350cd1-6737-44e5-809b-9c21ba403e85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23350cd1-6737-44e5-809b-9c21ba403e85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73c43698 2021-05-28T00:56:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23350cd1-6737-44e5-809b-9c21ba403e85 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23350cd1-6737-44e5-809b-9c21ba403e85 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:39.359Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1788 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:37.571Z InvocationID=2401156a-01b6-4109-964c-6c0d639efbdb LogTimestamp=2021-05-28T00:56:39.359199Z ResponseDesc=Created Exiting. 2021-05-28T00:56:39.359Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:39.368Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:39.368805Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:39.369Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:39.368805Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:39.369Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T00:56:39.379Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=092c0359-d1bb-4345-98f8-966ba6ad6456 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:39.379150Z InvocationID=462ee2b3-24c7-4875-82f1-bde1d3aa1dab LogTimestamp=2021-05-28T00:56:39.379192Z Entering 2021-05-28T00:56:39.379Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=092c0359-d1bb-4345-98f8-966ba6ad6456 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:39.379150Z InvocationID=462ee2b3-24c7-4875-82f1-bde1d3aa1dab LogTimestamp=2021-05-28T00:56:39.379192Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:39.477Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:39.379Z InvocationID=987b7aa4-4d5f-4aa4-a1c5-389c7fe9647a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T00:56:40.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b5e54f-adb1-4c2c-8f22-711d4827f9a7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:40.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b5e54f-adb1-4c2c-8f22-711d4827f9a7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e80b59c 2021-05-28T00:56:40.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b5e54f-adb1-4c2c-8f22-711d4827f9a7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:40.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02b5e54f-adb1-4c2c-8f22-711d4827f9a7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:41.571Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:39.379Z InvocationID=987b7aa4-4d5f-4aa4-a1c5-389c7fe9647a LogTimestamp=2021-05-28T00:56:41.571131Z ResponseDesc=Created Exiting. 2021-05-28T00:56:41.571Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:41.673Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:41.673259Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:41.673Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:41.673259Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:41.673Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T00:56:41.772Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=757149ce-a45e-4c0e-8b41-c1c2e88fe636 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:41.772402Z InvocationID=36a5a281-9a4b-4757-b678-3260ffcdb41b LogTimestamp=2021-05-28T00:56:41.772494Z Entering 2021-05-28T00:56:41.772Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=757149ce-a45e-4c0e-8b41-c1c2e88fe636 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:41.772402Z InvocationID=36a5a281-9a4b-4757-b678-3260ffcdb41b LogTimestamp=2021-05-28T00:56:41.772494Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:41.872Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:41.773Z InvocationID=807abf8b-29bc-40dc-be3d-69b7b88bc703 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T00:56:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e63f461-ef34-4ea7-a0b0-adfd70055a27 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e63f461-ef34-4ea7-a0b0-adfd70055a27 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14bf28bd 2021-05-28T00:56:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e63f461-ef34-4ea7-a0b0-adfd70055a27 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:43.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e63f461-ef34-4ea7-a0b0-adfd70055a27 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:44.071Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2298 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:41.773Z InvocationID=807abf8b-29bc-40dc-be3d-69b7b88bc703 LogTimestamp=2021-05-28T00:56:44.071656Z ResponseDesc=Created Exiting. 2021-05-28T00:56:44.072Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:44.172Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:44.171942Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:44.172Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:44.171942Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:44.172Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T00:56:44.275Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ad529f72-7b2d-4934-99e6-2d5058cfe2b8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:44.275402Z InvocationID=8344697e-7e74-42e5-95ba-87bbeaa4d215 LogTimestamp=2021-05-28T00:56:44.275455Z Entering 2021-05-28T00:56:44.275Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ad529f72-7b2d-4934-99e6-2d5058cfe2b8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:44.275402Z InvocationID=8344697e-7e74-42e5-95ba-87bbeaa4d215 LogTimestamp=2021-05-28T00:56:44.275455Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:44.370Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:44.275Z InvocationID=db54de50-7883-419b-965f-73d88cf94d8b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T00:56:46.060Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1784 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:44.275Z InvocationID=db54de50-7883-419b-965f-73d88cf94d8b LogTimestamp=2021-05-28T00:56:46.059895Z ResponseDesc=Created Exiting. 2021-05-28T00:56:46.060Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:46.163Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:46.163797Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:46.163Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:46.163797Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:46.164Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T00:56:46.263Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6573b44d-b5b2-43be-b4a2-fb119edb9934 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:46.263641Z InvocationID=7ddf067f-794e-4fd8-b404-38132f43c475 LogTimestamp=2021-05-28T00:56:46.263676Z Entering 2021-05-28T00:56:46.263Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6573b44d-b5b2-43be-b4a2-fb119edb9934 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:46.263641Z InvocationID=7ddf067f-794e-4fd8-b404-38132f43c475 LogTimestamp=2021-05-28T00:56:46.263676Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:46.364Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:46.264Z InvocationID=1b7b123d-aaa8-411c-86bc-fbea2ee66958 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T00:56:46.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58bde51d-ac55-4ec1-9487-d73816777020 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:46.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58bde51d-ac55-4ec1-9487-d73816777020 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18d3579f 2021-05-28T00:56:46.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58bde51d-ac55-4ec1-9487-d73816777020 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:46.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58bde51d-ac55-4ec1-9487-d73816777020 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:48.057Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1793 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:46.264Z InvocationID=1b7b123d-aaa8-411c-86bc-fbea2ee66958 LogTimestamp=2021-05-28T00:56:48.057131Z ResponseDesc=Created Exiting. 2021-05-28T00:56:48.057Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:48.082Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:48.082793Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:48.082Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:48.082793Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:48.083Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T00:56:48.172Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ff006c4c-1be7-433b-934e-43738204a018 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:48.171904Z InvocationID=f5ac0498-f206-458e-a8ad-2db620406d39 LogTimestamp=2021-05-28T00:56:48.171945Z Entering 2021-05-28T00:56:48.172Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ff006c4c-1be7-433b-934e-43738204a018 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:48.171904Z InvocationID=f5ac0498-f206-458e-a8ad-2db620406d39 LogTimestamp=2021-05-28T00:56:48.171945Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:48.180Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:48.172Z InvocationID=47b58364-4d23-48f4-8e1b-32fcc32b80fa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T00:56:49.665Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1492 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:48.172Z InvocationID=47b58364-4d23-48f4-8e1b-32fcc32b80fa LogTimestamp=2021-05-28T00:56:49.664908Z ResponseDesc=Created Exiting. 2021-05-28T00:56:49.665Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:49.762Z [qtp580718781-16] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:49.762746Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:49.762Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:49.762746Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:49.763Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T00:56:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce7b0238-37c0-4b05-b614-9f39ef3b02ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce7b0238-37c0-4b05-b614-9f39ef3b02ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4af1c2eb 2021-05-28T00:56:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce7b0238-37c0-4b05-b614-9f39ef3b02ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce7b0238-37c0-4b05-b614-9f39ef3b02ec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:49.771Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4fa9a7f0-f2b2-4532-8e8e-2edfa682bb2a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:49.771202Z InvocationID=90e30068-3fc2-4800-816d-8c2fec0aead6 LogTimestamp=2021-05-28T00:56:49.771261Z Entering 2021-05-28T00:56:49.771Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4fa9a7f0-f2b2-4532-8e8e-2edfa682bb2a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:49.771202Z InvocationID=90e30068-3fc2-4800-816d-8c2fec0aead6 LogTimestamp=2021-05-28T00:56:49.771261Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:49.873Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:49.771Z InvocationID=fdea8f08-168a-4f8b-8577-238ab9427a50 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T00:56:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:56:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:56:51.466Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1695 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:49.771Z InvocationID=fdea8f08-168a-4f8b-8577-238ab9427a50 LogTimestamp=2021-05-28T00:56:51.466020Z ResponseDesc=Created Exiting. 2021-05-28T00:56:51.466Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:51.474Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:51.474069Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:51.474Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:51.474069Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:51.474Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T00:56:51.563Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ec3a2574-f86a-4bd3-b694-47cdc18818c0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:51.563807Z InvocationID=a4a86280-743b-4ec5-be73-177c789de8fa LogTimestamp=2021-05-28T00:56:51.563840Z Entering 2021-05-28T00:56:51.564Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ec3a2574-f86a-4bd3-b694-47cdc18818c0 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:51.563807Z InvocationID=a4a86280-743b-4ec5-be73-177c789de8fa LogTimestamp=2021-05-28T00:56:51.563840Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:51.571Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:51.564Z InvocationID=07bcdf91-384f-4b5c-b0e8-32c438bcd849 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T00:56:52.126Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:56:53.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4093de5e-13dd-4174-ba9a-25e28960a195 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:53.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4093de5e-13dd-4174-ba9a-25e28960a195 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3126a27c 2021-05-28T00:56:53.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4093de5e-13dd-4174-ba9a-25e28960a195 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:53.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4093de5e-13dd-4174-ba9a-25e28960a195 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:53.372Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1807 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:51.564Z InvocationID=07bcdf91-384f-4b5c-b0e8-32c438bcd849 LogTimestamp=2021-05-28T00:56:53.371818Z ResponseDesc=Created Exiting. 2021-05-28T00:56:53.372Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:53.470Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:53.470763Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T00:56:53.470Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T00:56:53.470763Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:53.471Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T00:56:53.563Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=24eea055-fb52-43fe-8284-249c8a85cf0f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:53.563515Z InvocationID=70010b1c-d670-4565-bd92-280ec659df63 LogTimestamp=2021-05-28T00:56:53.563576Z Entering 2021-05-28T00:56:53.563Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=24eea055-fb52-43fe-8284-249c8a85cf0f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:53.563515Z InvocationID=70010b1c-d670-4565-bd92-280ec659df63 LogTimestamp=2021-05-28T00:56:53.563576Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T00:56:53.755Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:53.564Z InvocationID=c3671580-c5f5-46df-91cd-4c22ca9cf6dc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T00:56:55.472Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1908 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T00:56:53.564Z InvocationID=c3671580-c5f5-46df-91cd-4c22ca9cf6dc LogTimestamp=2021-05-28T00:56:55.472310Z ResponseDesc=Created Exiting. 2021-05-28T00:56:55.472Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T00:56:55.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=692ec8ec-fe19-436c-8002-8f783d60a4de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:55.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=692ec8ec-fe19-436c-8002-8f783d60a4de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c319e25 2021-05-28T00:56:55.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=692ec8ec-fe19-436c-8002-8f783d60a4de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:55.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=692ec8ec-fe19-436c-8002-8f783d60a4de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:56:55.868Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=326251fa-86a5-4a70-bb11-8b4150c89c3e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:55.868076Z InvocationID=91ea7217-be08-4d32-81af-7d3e99cbfec8 LogTimestamp=2021-05-28T00:56:55.868108Z Entering 2021-05-28T00:56:55.868Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=326251fa-86a5-4a70-bb11-8b4150c89c3e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:55.868076Z InvocationID=91ea7217-be08-4d32-81af-7d3e99cbfec8 LogTimestamp=2021-05-28T00:56:55.868108Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T00:56:55.959Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=91 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T00:56:55.868Z InvocationID=b928cbc7-5c24-476f-83dc-263175a761af LogTimestamp=2021-05-28T00:56:55.959075Z ResponseDesc=OK Exiting. 2021-05-28T00:56:55.959Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T00:56:56.474Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e4445618-4787-48cb-8676-0e138b7bbe9c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:56.474547Z InvocationID=bae7a7fd-489f-4d43-9987-138e4328aeb7 LogTimestamp=2021-05-28T00:56:56.474589Z Entering 2021-05-28T00:56:56.474Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e4445618-4787-48cb-8676-0e138b7bbe9c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:56.474547Z InvocationID=bae7a7fd-489f-4d43-9987-138e4328aeb7 LogTimestamp=2021-05-28T00:56:56.474589Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T00:56:56.768Z [qtp580718781-96] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:56:56.474Z InvocationID=489b94e2-4b3e-4008-95c6-15d1c8e2a23b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 291542377 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n291542377 100% \n 2021-05-28T00:56:56.774Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=300 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T00:56:56.474Z InvocationID=489b94e2-4b3e-4008-95c6-15d1c8e2a23b LogTimestamp=2021-05-28T00:56:56.774848Z ResponseDesc=OK Exiting. 2021-05-28T00:56:56.775Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T00:56:56.787Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3f0480a8-66c5-4245-8b90-2117eef9481e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:56.787118Z InvocationID=c510dd37-6796-4d2e-95a2-3a947f7af316 LogTimestamp=2021-05-28T00:56:56.787164Z Entering 2021-05-28T00:56:56.787Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3f0480a8-66c5-4245-8b90-2117eef9481e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:56:56.787118Z InvocationID=c510dd37-6796-4d2e-95a2-3a947f7af316 LogTimestamp=2021-05-28T00:56:56.787164Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:56:56.878Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:56:56.787Z InvocationID=e9ce31c1-9ce4-49f4-8f31-1587dbe85632 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:56:56.962Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:56:56.787Z InvocationID=e9ce31c1-9ce4-49f4-8f31-1587dbe85632 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:56:56.962Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:56:56.787Z InvocationID=e9ce31c1-9ce4-49f4-8f31-1587dbe85632 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:56:56.963Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=176 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:56:56.787Z InvocationID=e9ce31c1-9ce4-49f4-8f31-1587dbe85632 ErrorCode=500 LogTimestamp=2021-05-28T00:56:56.963024Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:56:56.963Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:56:58.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67ba2b6b-8b80-4406-9afd-664d9c7efd09 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:56:58.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67ba2b6b-8b80-4406-9afd-664d9c7efd09 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e5c4593 2021-05-28T00:56:58.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67ba2b6b-8b80-4406-9afd-664d9c7efd09 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:56:58.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67ba2b6b-8b80-4406-9afd-664d9c7efd09 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b559a613-7e41-48d9-935f-366c569b96ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b559a613-7e41-48d9-935f-366c569b96ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1afe2229 2021-05-28T00:57:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b559a613-7e41-48d9-935f-366c569b96ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:01.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b559a613-7e41-48d9-935f-366c569b96ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:04.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ff370c2-efc4-4e72-a14c-caa2a59055b5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:04.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ff370c2-efc4-4e72-a14c-caa2a59055b5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ecf4648 2021-05-28T00:57:04.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ff370c2-efc4-4e72-a14c-caa2a59055b5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:04.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ff370c2-efc4-4e72-a14c-caa2a59055b5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:57:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:57:07.071Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6ae370c8-5d11-4230-8d65-701fa0acbdee InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:07.071412Z InvocationID=201fdf5b-7dfc-4c5e-b3f2-58d91a146ddf LogTimestamp=2021-05-28T00:57:07.071458Z Entering 2021-05-28T00:57:07.071Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6ae370c8-5d11-4230-8d65-701fa0acbdee InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:07.071412Z InvocationID=201fdf5b-7dfc-4c5e-b3f2-58d91a146ddf LogTimestamp=2021-05-28T00:57:07.071458Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:57:07.158Z [qtp580718781-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:57:07.071Z InvocationID=5bf5bfba-0499-4e8b-aca5-779880dc238c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:57:07.168Z [qtp580718781-18] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:07.071Z InvocationID=5bf5bfba-0499-4e8b-aca5-779880dc238c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:57:07.168Z [qtp580718781-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:07.071Z InvocationID=5bf5bfba-0499-4e8b-aca5-779880dc238c ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:57:07.169Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=97 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:57:07.071Z InvocationID=5bf5bfba-0499-4e8b-aca5-779880dc238c ErrorCode=500 LogTimestamp=2021-05-28T00:57:07.168943Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:57:07.169Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:57:07.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7e6706d-418c-4911-9a9b-9967b69511e8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:07.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7e6706d-418c-4911-9a9b-9967b69511e8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e5f2302 2021-05-28T00:57:07.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7e6706d-418c-4911-9a9b-9967b69511e8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:07.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7e6706d-418c-4911-9a9b-9967b69511e8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bc06494-4005-4ca9-93e2-525c3d15a54f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bc06494-4005-4ca9-93e2-525c3d15a54f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72139787 2021-05-28T00:57:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bc06494-4005-4ca9-93e2-525c3d15a54f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bc06494-4005-4ca9-93e2-525c3d15a54f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3242478-e31d-4f92-8bb9-8fc845fc690b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3242478-e31d-4f92-8bb9-8fc845fc690b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19970c86 2021-05-28T00:57:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3242478-e31d-4f92-8bb9-8fc845fc690b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3242478-e31d-4f92-8bb9-8fc845fc690b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8b5ab52-b1dd-4990-938c-322e4be8de08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8b5ab52-b1dd-4990-938c-322e4be8de08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d8ee4fa 2021-05-28T00:57:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8b5ab52-b1dd-4990-938c-322e4be8de08 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8b5ab52-b1dd-4990-938c-322e4be8de08 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:17.276Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9fbfd440-af3c-4160-9fcf-7661e874b7e3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:17.276487Z InvocationID=1dd3bfbd-5df1-4a6c-8fd2-f0dc81dc7217 LogTimestamp=2021-05-28T00:57:17.276526Z Entering 2021-05-28T00:57:17.276Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9fbfd440-af3c-4160-9fcf-7661e874b7e3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:17.276487Z InvocationID=1dd3bfbd-5df1-4a6c-8fd2-f0dc81dc7217 LogTimestamp=2021-05-28T00:57:17.276526Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:57:17.362Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:57:17.276Z InvocationID=30c87cd5-b5ba-4b18-b11d-9f1603f4469c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:57:17.459Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:17.276Z InvocationID=30c87cd5-b5ba-4b18-b11d-9f1603f4469c ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:57:17.460Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:17.276Z InvocationID=30c87cd5-b5ba-4b18-b11d-9f1603f4469c ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:57:17.460Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=184 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:57:17.276Z InvocationID=30c87cd5-b5ba-4b18-b11d-9f1603f4469c ErrorCode=500 LogTimestamp=2021-05-28T00:57:17.460524Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:57:17.460Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:57:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbf9fb4c-da68-46e9-bc36-f036d42a6f15 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbf9fb4c-da68-46e9-bc36-f036d42a6f15 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4afe9f7a 2021-05-28T00:57:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbf9fb4c-da68-46e9-bc36-f036d42a6f15 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbf9fb4c-da68-46e9-bc36-f036d42a6f15 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:57:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:57:22.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcbb25e8-2e25-49bc-b24d-e05827aae7f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:22.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcbb25e8-2e25-49bc-b24d-e05827aae7f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@243f960d 2021-05-28T00:57:22.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcbb25e8-2e25-49bc-b24d-e05827aae7f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:22.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcbb25e8-2e25-49bc-b24d-e05827aae7f3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:25.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8f80765-265c-4e30-ad11-e9644c58fc66 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:25.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8f80765-265c-4e30-ad11-e9644c58fc66 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ed5d30 2021-05-28T00:57:25.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8f80765-265c-4e30-ad11-e9644c58fc66 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:25.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8f80765-265c-4e30-ad11-e9644c58fc66 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:27.573Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7e083210-b7b0-4dc1-b128-bb5158529199 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:27.573539Z InvocationID=b86a185b-6af8-4a15-a2b6-aea254ddcf7e LogTimestamp=2021-05-28T00:57:27.573579Z Entering 2021-05-28T00:57:27.573Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7e083210-b7b0-4dc1-b128-bb5158529199 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:27.573539Z InvocationID=b86a185b-6af8-4a15-a2b6-aea254ddcf7e LogTimestamp=2021-05-28T00:57:27.573579Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:57:27.676Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:57:27.574Z InvocationID=65df8aff-3269-4c77-b9fa-3b18fdf78a2d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:57:27.768Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:27.574Z InvocationID=65df8aff-3269-4c77-b9fa-3b18fdf78a2d ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:57:27.768Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:27.574Z InvocationID=65df8aff-3269-4c77-b9fa-3b18fdf78a2d ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:57:27.769Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=195 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:57:27.574Z InvocationID=65df8aff-3269-4c77-b9fa-3b18fdf78a2d ErrorCode=500 LogTimestamp=2021-05-28T00:57:27.769730Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:57:27.770Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:57:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44e0216-0bbe-4c2f-a916-cac22a042838 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44e0216-0bbe-4c2f-a916-cac22a042838 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f4ec043 2021-05-28T00:57:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44e0216-0bbe-4c2f-a916-cac22a042838 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44e0216-0bbe-4c2f-a916-cac22a042838 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:31.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=451f7f9d-8b5f-4e97-ad3b-99d616e7f204 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:31.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=451f7f9d-8b5f-4e97-ad3b-99d616e7f204 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31f58b9f 2021-05-28T00:57:31.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=451f7f9d-8b5f-4e97-ad3b-99d616e7f204 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:31.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=451f7f9d-8b5f-4e97-ad3b-99d616e7f204 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bdd0121-8110-42c0-a6f7-8a682ee71492 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bdd0121-8110-42c0-a6f7-8a682ee71492 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69a2ec52 2021-05-28T00:57:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bdd0121-8110-42c0-a6f7-8a682ee71492 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bdd0121-8110-42c0-a6f7-8a682ee71492 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:57:35.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:57:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:57:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:57:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:57:37.964Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0b84a6d5-85a6-4037-8e85-1cd8b81da144 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:37.963683Z InvocationID=1d8067dd-5724-4097-aff7-6e6fac0e37ea LogTimestamp=2021-05-28T00:57:37.963765Z Entering 2021-05-28T00:57:37.964Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0b84a6d5-85a6-4037-8e85-1cd8b81da144 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:37.963683Z InvocationID=1d8067dd-5724-4097-aff7-6e6fac0e37ea LogTimestamp=2021-05-28T00:57:37.963765Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:57:37.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e6f51d0-4775-4263-968c-d39b36accce6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:37.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e6f51d0-4775-4263-968c-d39b36accce6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c9388b 2021-05-28T00:57:37.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e6f51d0-4775-4263-968c-d39b36accce6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:37.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e6f51d0-4775-4263-968c-d39b36accce6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:38.070Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:57:37.964Z InvocationID=8ffda03e-dbd0-4774-b392-5606ceae1adb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:57:38.088Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:37.964Z InvocationID=8ffda03e-dbd0-4774-b392-5606ceae1adb ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:57:38.088Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:37.964Z InvocationID=8ffda03e-dbd0-4774-b392-5606ceae1adb ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:57:38.089Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=125 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:57:37.964Z InvocationID=8ffda03e-dbd0-4774-b392-5606ceae1adb ErrorCode=500 LogTimestamp=2021-05-28T00:57:38.089123Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:57:38.089Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:57:40.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b97d9a1-db21-4963-aa6f-50616802c2a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:40.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b97d9a1-db21-4963-aa6f-50616802c2a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ffcacc9 2021-05-28T00:57:40.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b97d9a1-db21-4963-aa6f-50616802c2a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:40.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b97d9a1-db21-4963-aa6f-50616802c2a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:43.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfa65a5-ce58-4f8e-97a6-3ff0da3070a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:43.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfa65a5-ce58-4f8e-97a6-3ff0da3070a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@624879da 2021-05-28T00:57:43.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfa65a5-ce58-4f8e-97a6-3ff0da3070a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:43.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfa65a5-ce58-4f8e-97a6-3ff0da3070a8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:47.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f99444-27f5-4714-ac47-94f053391bce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:47.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f99444-27f5-4714-ac47-94f053391bce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cf6c9bd 2021-05-28T00:57:47.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f99444-27f5-4714-ac47-94f053391bce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:47.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19f99444-27f5-4714-ac47-94f053391bce PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:49.876Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5334cb4d-3494-43f6-9acf-8dd5c53de98d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:49.876195Z InvocationID=d9224b86-7b98-4b5d-8ee6-4152298a45c0 LogTimestamp=2021-05-28T00:57:49.876270Z Entering 2021-05-28T00:57:49.876Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5334cb4d-3494-43f6-9acf-8dd5c53de98d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:57:49.876195Z InvocationID=d9224b86-7b98-4b5d-8ee6-4152298a45c0 LogTimestamp=2021-05-28T00:57:49.876270Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:57:49.971Z [qtp580718781-15] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:57:49.876Z InvocationID=045b9d01-0234-44da-b200-11a0ddd99bfa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:57:50.067Z [qtp580718781-15] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:49.876Z InvocationID=045b9d01-0234-44da-b200-11a0ddd99bfa ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:57:50.067Z [qtp580718781-15] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:57:49.876Z InvocationID=045b9d01-0234-44da-b200-11a0ddd99bfa ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:57:50.068Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=192 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:57:49.876Z InvocationID=045b9d01-0234-44da-b200-11a0ddd99bfa ErrorCode=500 LogTimestamp=2021-05-28T00:57:50.068373Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:57:50.068Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:57:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:57:50.517Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:57:50.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c23352b-ebf5-4d50-9937-4edd39ab14c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:50.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c23352b-ebf5-4d50-9937-4edd39ab14c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53b4a7c8 2021-05-28T00:57:50.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c23352b-ebf5-4d50-9937-4edd39ab14c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:50.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c23352b-ebf5-4d50-9937-4edd39ab14c7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:52.158Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:57:52.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d492a2cf-ca5c-4041-b536-ae4450fe3bd8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:52.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d492a2cf-ca5c-4041-b536-ae4450fe3bd8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49b4d393 2021-05-28T00:57:52.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d492a2cf-ca5c-4041-b536-ae4450fe3bd8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:52.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d492a2cf-ca5c-4041-b536-ae4450fe3bd8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:55.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce74d60b-bbfe-4a29-affc-49a4e800ba3d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:55.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce74d60b-bbfe-4a29-affc-49a4e800ba3d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6813e5d6 2021-05-28T00:57:55.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce74d60b-bbfe-4a29-affc-49a4e800ba3d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:55.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce74d60b-bbfe-4a29-affc-49a4e800ba3d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:57:58.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43b3c6-742a-4938-a1aa-805eba15efe6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:57:58.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43b3c6-742a-4938-a1aa-805eba15efe6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42026248 2021-05-28T00:57:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43b3c6-742a-4938-a1aa-805eba15efe6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:57:58.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43b3c6-742a-4938-a1aa-805eba15efe6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:00.170Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=43d803f3-0818-4cf9-b957-0b752710eb54 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:00.169957Z InvocationID=13b67664-ecbc-4a0a-9400-c8b376847a5a LogTimestamp=2021-05-28T00:58:00.170085Z Entering 2021-05-28T00:58:00.170Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=43d803f3-0818-4cf9-b957-0b752710eb54 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:00.169957Z InvocationID=13b67664-ecbc-4a0a-9400-c8b376847a5a LogTimestamp=2021-05-28T00:58:00.170085Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:58:00.404Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:58:00.170Z InvocationID=fb20e332-ec2f-4714-b466-3b0bfd749679 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:58:00.473Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:00.170Z InvocationID=fb20e332-ec2f-4714-b466-3b0bfd749679 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:58:00.473Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:00.170Z InvocationID=fb20e332-ec2f-4714-b466-3b0bfd749679 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:58:00.473Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=303 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:58:00.170Z InvocationID=fb20e332-ec2f-4714-b466-3b0bfd749679 ErrorCode=500 LogTimestamp=2021-05-28T00:58:00.473724Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:58:00.474Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:58:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71540ba7-c6c1-4c2d-9e08-55473f510677 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71540ba7-c6c1-4c2d-9e08-55473f510677 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@347e0727 2021-05-28T00:58:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71540ba7-c6c1-4c2d-9e08-55473f510677 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71540ba7-c6c1-4c2d-9e08-55473f510677 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:04.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e950569-fb32-435b-87dc-6a12a9cf86d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:04.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e950569-fb32-435b-87dc-6a12a9cf86d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b397270 2021-05-28T00:58:04.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e950569-fb32-435b-87dc-6a12a9cf86d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:04.985Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e950569-fb32-435b-87dc-6a12a9cf86d1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:58:05.632Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:58:07.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9af1d70b-ffc3-4e22-8960-dd09a468ca33 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:07.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9af1d70b-ffc3-4e22-8960-dd09a468ca33 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c730194 2021-05-28T00:58:07.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9af1d70b-ffc3-4e22-8960-dd09a468ca33 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:07.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9af1d70b-ffc3-4e22-8960-dd09a468ca33 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:10.761Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=65caec51-fe43-4856-aedc-6b2028fc5a20 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:10.761233Z InvocationID=260319a6-3650-46ae-ae84-c4eef8cef9a9 LogTimestamp=2021-05-28T00:58:10.761290Z Entering 2021-05-28T00:58:10.761Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=65caec51-fe43-4856-aedc-6b2028fc5a20 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:10.761233Z InvocationID=260319a6-3650-46ae-ae84-c4eef8cef9a9 LogTimestamp=2021-05-28T00:58:10.761290Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:58:10.867Z [qtp580718781-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:58:10.761Z InvocationID=781eb317-ad60-415f-9e1f-889f107e7635 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:58:10.979Z [qtp580718781-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:10.761Z InvocationID=781eb317-ad60-415f-9e1f-889f107e7635 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:58:10.979Z [qtp580718781-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:10.761Z InvocationID=781eb317-ad60-415f-9e1f-889f107e7635 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:58:10.980Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=219 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:58:10.761Z InvocationID=781eb317-ad60-415f-9e1f-889f107e7635 ErrorCode=500 LogTimestamp=2021-05-28T00:58:10.980211Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:58:10.980Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:58:11.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75be3aa6-8cf9-4565-a800-f1e1423ef3fa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:11.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75be3aa6-8cf9-4565-a800-f1e1423ef3fa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d322670 2021-05-28T00:58:11.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75be3aa6-8cf9-4565-a800-f1e1423ef3fa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:11.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75be3aa6-8cf9-4565-a800-f1e1423ef3fa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:13.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e67c3c1c-bebe-4959-b02c-f1ccdd6a4098 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:13.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e67c3c1c-bebe-4959-b02c-f1ccdd6a4098 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@393141ce 2021-05-28T00:58:13.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e67c3c1c-bebe-4959-b02c-f1ccdd6a4098 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:13.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e67c3c1c-bebe-4959-b02c-f1ccdd6a4098 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:16.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf259af-681e-437a-aa57-45901ca6a48d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:16.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf259af-681e-437a-aa57-45901ca6a48d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@298b0e6a 2021-05-28T00:58:16.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf259af-681e-437a-aa57-45901ca6a48d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:16.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf259af-681e-437a-aa57-45901ca6a48d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:58:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:58:20.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cf984fb-6e63-4ac1-9a6c-0c029836afdc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:20.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cf984fb-6e63-4ac1-9a6c-0c029836afdc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dc0c56e 2021-05-28T00:58:20.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cf984fb-6e63-4ac1-9a6c-0c029836afdc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:20.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cf984fb-6e63-4ac1-9a6c-0c029836afdc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:21.080Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ab9c6771-d0b7-4feb-8731-911cb5db6230 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:21.080472Z InvocationID=df5027d9-8f05-40d0-9338-92b62d6be8cd LogTimestamp=2021-05-28T00:58:21.080583Z Entering 2021-05-28T00:58:21.081Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ab9c6771-d0b7-4feb-8731-911cb5db6230 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:21.080472Z InvocationID=df5027d9-8f05-40d0-9338-92b62d6be8cd LogTimestamp=2021-05-28T00:58:21.080583Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:58:21.182Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:58:21.081Z InvocationID=7afe4747-7311-4c70-a7d1-4773150b5964 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:58:21.267Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:21.081Z InvocationID=7afe4747-7311-4c70-a7d1-4773150b5964 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:58:21.268Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:21.081Z InvocationID=7afe4747-7311-4c70-a7d1-4773150b5964 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:58:21.268Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=187 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:58:21.081Z InvocationID=7afe4747-7311-4c70-a7d1-4773150b5964 ErrorCode=500 LogTimestamp=2021-05-28T00:58:21.268494Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:58:21.268Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:58:23.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:23.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71ad7100 2021-05-28T00:58:23.059Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:23.059Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:26.065Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:58:26.067Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:58:26.067Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:26.067Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4ad50afc-cc70-4ade-86bf-91eb8823b4a9 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:58:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T00:58:27.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8644cb6c-657a-4c7e-8d78-74cb30216a48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:27.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8644cb6c-657a-4c7e-8d78-74cb30216a48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42a161ca 2021-05-28T00:58:27.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8644cb6c-657a-4c7e-8d78-74cb30216a48 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:27.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8644cb6c-657a-4c7e-8d78-74cb30216a48 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:27.771Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=8644cb6c-657a-4c7e-8d78-74cb30216a48 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:27.771Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=8644cb6c-657a-4c7e-8d78-74cb30216a48 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:32.182Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T00:58:32.185Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a30920fe-c600-476c-be31-e8bc29669944 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:32.185045Z InvocationID=677683a4-2333-41bb-89c5-d0afd21c5ac9 LogTimestamp=2021-05-28T00:58:32.185125Z Entering 2021-05-28T00:58:32.185Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a30920fe-c600-476c-be31-e8bc29669944 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:32.185045Z InvocationID=677683a4-2333-41bb-89c5-d0afd21c5ac9 LogTimestamp=2021-05-28T00:58:32.185125Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:58:32.313Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:58:32.185Z InvocationID=e3ac4f8c-3671-4031-93a5-e9fe6beb5320 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:58:32.371Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:32.185Z InvocationID=e3ac4f8c-3671-4031-93a5-e9fe6beb5320 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:58:32.371Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:32.185Z InvocationID=e3ac4f8c-3671-4031-93a5-e9fe6beb5320 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:58:32.371Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=186 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:58:32.185Z InvocationID=e3ac4f8c-3671-4031-93a5-e9fe6beb5320 ErrorCode=500 LogTimestamp=2021-05-28T00:58:32.371589Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:58:32.371Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:58:32.388Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f058affe-2ddd-444e-ac72-670736abd250 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:32.388Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f058affe-2ddd-444e-ac72-670736abd250 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28d88704 2021-05-28T00:58:32.388Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f058affe-2ddd-444e-ac72-670736abd250 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:32.388Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f058affe-2ddd-444e-ac72-670736abd250 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:32.859Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f058affe-2ddd-444e-ac72-670736abd250 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:32.859Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=f058affe-2ddd-444e-ac72-670736abd250 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T00:58:34.156Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1a20dc0-d5d2-4542-964e-322ada49bdf1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:34.156Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1a20dc0-d5d2-4542-964e-322ada49bdf1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3267c66e 2021-05-28T00:58:34.156Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1a20dc0-d5d2-4542-964e-322ada49bdf1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:34.156Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1a20dc0-d5d2-4542-964e-322ada49bdf1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:34.671Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c1a20dc0-d5d2-4542-964e-322ada49bdf1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:34.671Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c1a20dc0-d5d2-4542-964e-322ada49bdf1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:58:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:58:35.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:35.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b015ed9 2021-05-28T00:58:35.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:35.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:58:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:58:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:58:38.959Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:58:38.959Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:58:38.959Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:38.959Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ae97a7d7-76ac-4661-9bee-95019a8ed23c ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:41.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37cbc1ec-43e7-4f0e-b3f7-c24a42c3925d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:41.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37cbc1ec-43e7-4f0e-b3f7-c24a42c3925d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e219b9c 2021-05-28T00:58:41.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37cbc1ec-43e7-4f0e-b3f7-c24a42c3925d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:41.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37cbc1ec-43e7-4f0e-b3f7-c24a42c3925d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:41.977Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=37cbc1ec-43e7-4f0e-b3f7-c24a42c3925d ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:41.977Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=37cbc1ec-43e7-4f0e-b3f7-c24a42c3925d ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:58:42.578Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=df6954e7-c4b6-4e3a-96f6-06ea8621f09b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:42.578456Z InvocationID=20ac0b2f-e46f-4671-817a-ac5f5e7fde20 LogTimestamp=2021-05-28T00:58:42.578623Z Entering 2021-05-28T00:58:42.579Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=df6954e7-c4b6-4e3a-96f6-06ea8621f09b InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.182 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.182 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T00:58:42.578456Z InvocationID=20ac0b2f-e46f-4671-817a-ac5f5e7fde20 LogTimestamp=2021-05-28T00:58:42.578623Z localAddr=10.242.102.221 remoteAddr=10.242.170.182 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T00:58:42.689Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T00:58:42.579Z InvocationID=ea8f8158-7b4e-4582-a137-372aa1eea1c3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T00:58:42.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0749807-6a59-4241-8744-b2c40c8f28e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:42.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0749807-6a59-4241-8744-b2c40c8f28e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ef53376 2021-05-28T00:58:42.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0749807-6a59-4241-8744-b2c40c8f28e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:42.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0749807-6a59-4241-8744-b2c40c8f28e6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:42.865Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:42.579Z InvocationID=ea8f8158-7b4e-4582-a137-372aa1eea1c3 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T00:58:42.866Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T00:58:42.579Z InvocationID=ea8f8158-7b4e-4582-a137-372aa1eea1c3 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T00:58:42.866Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=287 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T00:58:42.579Z InvocationID=ea8f8158-7b4e-4582-a137-372aa1eea1c3 ErrorCode=500 LogTimestamp=2021-05-28T00:58:42.866410Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T00:58:42.866Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T00:58:44.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2199b0-9400-4296-afa1-8738acaf7794 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:44.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2199b0-9400-4296-afa1-8738acaf7794 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ff34fe7 2021-05-28T00:58:44.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2199b0-9400-4296-afa1-8738acaf7794 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:44.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2199b0-9400-4296-afa1-8738acaf7794 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a050d413-874d-45f1-bed5-cb01f3c3e9e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a050d413-874d-45f1-bed5-cb01f3c3e9e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d8ccafe 2021-05-28T00:58:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a050d413-874d-45f1-bed5-cb01f3c3e9e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a050d413-874d-45f1-bed5-cb01f3c3e9e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:49.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daac8e04-3287-4418-8fb6-8788cf896295 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:49.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daac8e04-3287-4418-8fb6-8788cf896295 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11c20ea1 2021-05-28T00:58:49.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daac8e04-3287-4418-8fb6-8788cf896295 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:49.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daac8e04-3287-4418-8fb6-8788cf896295 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:58:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:58:52.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:58:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1dc7d18-08a8-4044-a949-a983192054da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1dc7d18-08a8-4044-a949-a983192054da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56ebe16c 2021-05-28T00:58:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1dc7d18-08a8-4044-a949-a983192054da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:53.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1dc7d18-08a8-4044-a949-a983192054da PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:55.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6004d863-60ed-4e11-8949-5166bfc2ad9a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:55.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6004d863-60ed-4e11-8949-5166bfc2ad9a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53e21021 2021-05-28T00:58:55.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6004d863-60ed-4e11-8949-5166bfc2ad9a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:55.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6004d863-60ed-4e11-8949-5166bfc2ad9a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:58:58.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c07b2d-d3d3-4c28-9bbc-e00eab385206 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:58:58.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c07b2d-d3d3-4c28-9bbc-e00eab385206 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28c2eb99 2021-05-28T00:58:58.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c07b2d-d3d3-4c28-9bbc-e00eab385206 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:58:58.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c07b2d-d3d3-4c28-9bbc-e00eab385206 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:01.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06806bae-8d78-40bb-b3a3-304595e74dea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:01.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06806bae-8d78-40bb-b3a3-304595e74dea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29745f26 2021-05-28T00:59:01.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06806bae-8d78-40bb-b3a3-304595e74dea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:01.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06806bae-8d78-40bb-b3a3-304595e74dea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:05.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2954a593-7b35-481c-9bf0-704c7ac0e8f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:05.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2954a593-7b35-481c-9bf0-704c7ac0e8f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@209d3be5 2021-05-28T00:59:05.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2954a593-7b35-481c-9bf0-704c7ac0e8f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:05.088Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2954a593-7b35-481c-9bf0-704c7ac0e8f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:59:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:59:07.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e50a15-e764-405d-bba8-be1941dd0495 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:07.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e50a15-e764-405d-bba8-be1941dd0495 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@633d44cf 2021-05-28T00:59:07.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e50a15-e764-405d-bba8-be1941dd0495 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:07.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31e50a15-e764-405d-bba8-be1941dd0495 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:10.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94b15175-b434-4f2f-82bf-c6b017b93b40 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:10.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94b15175-b434-4f2f-82bf-c6b017b93b40 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c9ce512 2021-05-28T00:59:10.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94b15175-b434-4f2f-82bf-c6b017b93b40 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:10.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94b15175-b434-4f2f-82bf-c6b017b93b40 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8649ba83-5e2a-4b22-b1df-3902ac8fca46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8649ba83-5e2a-4b22-b1df-3902ac8fca46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@579a1cc8 2021-05-28T00:59:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8649ba83-5e2a-4b22-b1df-3902ac8fca46 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8649ba83-5e2a-4b22-b1df-3902ac8fca46 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:16.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c26d4b61-d4d6-43eb-9d6b-874fca785c4a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:16.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c26d4b61-d4d6-43eb-9d6b-874fca785c4a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e1147cd 2021-05-28T00:59:16.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c26d4b61-d4d6-43eb-9d6b-874fca785c4a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:16.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c26d4b61-d4d6-43eb-9d6b-874fca785c4a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:19.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c4052b-26ea-4ead-b87d-ed1cdb9cfc68 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:19.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c4052b-26ea-4ead-b87d-ed1cdb9cfc68 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@170373fc 2021-05-28T00:59:19.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c4052b-26ea-4ead-b87d-ed1cdb9cfc68 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:19.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c4052b-26ea-4ead-b87d-ed1cdb9cfc68 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:59:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:59:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28cf9135-4eb3-47d2-8508-d8ffc30a15c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28cf9135-4eb3-47d2-8508-d8ffc30a15c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c8ca07b 2021-05-28T00:59:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28cf9135-4eb3-47d2-8508-d8ffc30a15c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28cf9135-4eb3-47d2-8508-d8ffc30a15c5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8c27b8-3ead-4255-99ed-a0ecacc79353 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8c27b8-3ead-4255-99ed-a0ecacc79353 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b77957e 2021-05-28T00:59:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8c27b8-3ead-4255-99ed-a0ecacc79353 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8c27b8-3ead-4255-99ed-a0ecacc79353 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:29.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e9eedc-07e0-4d3f-bba8-670afd42bfeb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:29.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e9eedc-07e0-4d3f-bba8-670afd42bfeb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f4c90f2 2021-05-28T00:59:29.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e9eedc-07e0-4d3f-bba8-670afd42bfeb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:29.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0e9eedc-07e0-4d3f-bba8-670afd42bfeb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:31.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b203d71d-61b2-4152-9455-27b139b31d51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:31.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b203d71d-61b2-4152-9455-27b139b31d51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@335c9874 2021-05-28T00:59:31.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b203d71d-61b2-4152-9455-27b139b31d51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:31.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b203d71d-61b2-4152-9455-27b139b31d51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:35.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26de576e-6084-4796-a748-dce0eae0e9c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:35.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26de576e-6084-4796-a748-dce0eae0e9c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a1a791c 2021-05-28T00:59:35.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26de576e-6084-4796-a748-dce0eae0e9c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:35.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26de576e-6084-4796-a748-dce0eae0e9c2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:59:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:59:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T00:59:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T00:59:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T00:59:37.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c0c8e5e-28c4-40f5-a43e-b4b9db900c46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:37.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c0c8e5e-28c4-40f5-a43e-b4b9db900c46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d51cf44 2021-05-28T00:59:37.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c0c8e5e-28c4-40f5-a43e-b4b9db900c46 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:37.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c0c8e5e-28c4-40f5-a43e-b4b9db900c46 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:41.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899bc295-554a-47e1-b6e5-4a76aa5d85d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:41.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899bc295-554a-47e1-b6e5-4a76aa5d85d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36c1191c 2021-05-28T00:59:41.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899bc295-554a-47e1-b6e5-4a76aa5d85d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:41.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=899bc295-554a-47e1-b6e5-4a76aa5d85d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:43.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7ac67ba-cf49-4974-ac75-2d7d07b98637 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:43.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7ac67ba-cf49-4974-ac75-2d7d07b98637 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6109a888 2021-05-28T00:59:43.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7ac67ba-cf49-4974-ac75-2d7d07b98637 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:43.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7ac67ba-cf49-4974-ac75-2d7d07b98637 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:46.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2103d53-1150-42d8-b730-aedf0081e736 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:46.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2103d53-1150-42d8-b730-aedf0081e736 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4872ff2c 2021-05-28T00:59:46.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2103d53-1150-42d8-b730-aedf0081e736 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:46.789Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2103d53-1150-42d8-b730-aedf0081e736 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea79b1f-afc6-40b6-b5f0-b8257de0044d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea79b1f-afc6-40b6-b5f0-b8257de0044d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b3a3289 2021-05-28T00:59:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea79b1f-afc6-40b6-b5f0-b8257de0044d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea79b1f-afc6-40b6-b5f0-b8257de0044d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T00:59:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:59:52.109Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T00:59:53.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f93d1a3b-591f-42bb-b4dc-5720df19a8f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:53.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f93d1a3b-591f-42bb-b4dc-5720df19a8f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@158dc70b 2021-05-28T00:59:53.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f93d1a3b-591f-42bb-b4dc-5720df19a8f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:53.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f93d1a3b-591f-42bb-b4dc-5720df19a8f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:56.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T00:59:56.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@396c70da 2021-05-28T00:59:56.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T00:59:56.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T00:59:59.075Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T00:59:59.075Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T00:59:59.076Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:59:59.076Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ec4f036c-37a1-49ec-b3e6-ec6feb612bf1 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T00:59:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T00:59:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:00:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:00:04.083Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:00:04.157Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:00:05.477Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:00:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:00:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:00:05.992Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d6043d-4de2-4ef9-8e87-fcca27c8f9a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:05.992Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d6043d-4de2-4ef9-8e87-fcca27c8f9a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66386344 2021-05-28T01:00:05.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d6043d-4de2-4ef9-8e87-fcca27c8f9a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:05.993Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d6043d-4de2-4ef9-8e87-fcca27c8f9a3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-05-28T01:00:06.479Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=12d6043d-4de2-4ef9-8e87-fcca27c8f9a3 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:00:06.479Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=12d6043d-4de2-4ef9-8e87-fcca27c8f9a3 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:00:07.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1371a291-808f-40ec-9b28-c5e03e6c1089 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:07.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1371a291-808f-40ec-9b28-c5e03e6c1089 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67228269 2021-05-28T01:00:07.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1371a291-808f-40ec-9b28-c5e03e6c1089 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:07.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1371a291-808f-40ec-9b28-c5e03e6c1089 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:08.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c28a110f-3b85-4dbf-a377-4e92231f95b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:08.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c28a110f-3b85-4dbf-a377-4e92231f95b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d92d4f7 2021-05-28T01:00:08.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c28a110f-3b85-4dbf-a377-4e92231f95b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:08.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c28a110f-3b85-4dbf-a377-4e92231f95b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:09.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295e0dce-13e5-42d7-ba79-10adaea91686 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:09.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295e0dce-13e5-42d7-ba79-10adaea91686 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1895a242 2021-05-28T01:00:09.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295e0dce-13e5-42d7-ba79-10adaea91686 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:09.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295e0dce-13e5-42d7-ba79-10adaea91686 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:11.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daab5dec-e9dc-4d40-934c-61eef2727297 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:11.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daab5dec-e9dc-4d40-934c-61eef2727297 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61dbad0b 2021-05-28T01:00:11.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daab5dec-e9dc-4d40-934c-61eef2727297 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:11.181Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=daab5dec-e9dc-4d40-934c-61eef2727297 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:14.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5bea451-235e-49b8-9583-47a7d62433e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:14.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5bea451-235e-49b8-9583-47a7d62433e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12194a93 2021-05-28T01:00:14.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5bea451-235e-49b8-9583-47a7d62433e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:14.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5bea451-235e-49b8-9583-47a7d62433e4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:17.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95924d1b-07a3-4ce2-9696-aabc432dfea3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:17.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95924d1b-07a3-4ce2-9696-aabc432dfea3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3907114b 2021-05-28T01:00:17.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95924d1b-07a3-4ce2-9696-aabc432dfea3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:17.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95924d1b-07a3-4ce2-9696-aabc432dfea3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:20.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9f3b960-af5d-4ad5-af87-309e977bb782 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:20.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9f3b960-af5d-4ad5-af87-309e977bb782 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cb5239b 2021-05-28T01:00:20.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9f3b960-af5d-4ad5-af87-309e977bb782 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:20.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9f3b960-af5d-4ad5-af87-309e977bb782 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:00:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:00:22.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f015399b-d1fa-4dd4-afb7-53a07854f00c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:22.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f015399b-d1fa-4dd4-afb7-53a07854f00c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@303a64f3 2021-05-28T01:00:22.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f015399b-d1fa-4dd4-afb7-53a07854f00c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:22.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f015399b-d1fa-4dd4-afb7-53a07854f00c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:25.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0840fb14-274f-49fd-8f50-b94052eed5fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:25.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0840fb14-274f-49fd-8f50-b94052eed5fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68affdb5 2021-05-28T01:00:25.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0840fb14-274f-49fd-8f50-b94052eed5fc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:25.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0840fb14-274f-49fd-8f50-b94052eed5fc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa0d4a55-adb0-4a22-ad91-1188f914d02d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa0d4a55-adb0-4a22-ad91-1188f914d02d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@692324a2 2021-05-28T01:00:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa0d4a55-adb0-4a22-ad91-1188f914d02d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa0d4a55-adb0-4a22-ad91-1188f914d02d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:32.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e0f6a1-70d0-4523-a1f9-1b4093ed31bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:32.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e0f6a1-70d0-4523-a1f9-1b4093ed31bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c2f6be6 2021-05-28T01:00:32.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e0f6a1-70d0-4523-a1f9-1b4093ed31bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:32.469Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9e0f6a1-70d0-4523-a1f9-1b4093ed31bc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:35.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a079e535-6619-4b9c-ba17-b823af7c21dd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:35.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a079e535-6619-4b9c-ba17-b823af7c21dd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fd1836c 2021-05-28T01:00:35.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a079e535-6619-4b9c-ba17-b823af7c21dd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:35.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a079e535-6619-4b9c-ba17-b823af7c21dd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:00:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:00:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:00:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:00:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:00:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d98cb3a-f7a8-4fbc-b7f0-0a75ef35dbbc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d98cb3a-f7a8-4fbc-b7f0-0a75ef35dbbc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38048176 2021-05-28T01:00:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d98cb3a-f7a8-4fbc-b7f0-0a75ef35dbbc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d98cb3a-f7a8-4fbc-b7f0-0a75ef35dbbc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:41.282Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9270b44c-fa78-4849-9636-ae2aeb75d7d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:41.282Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9270b44c-fa78-4849-9636-ae2aeb75d7d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48f2b4e0 2021-05-28T01:00:41.282Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9270b44c-fa78-4849-9636-ae2aeb75d7d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:41.282Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9270b44c-fa78-4849-9636-ae2aeb75d7d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:43.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6642be3b-2b91-403e-b880-72af96e2a75a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:43.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6642be3b-2b91-403e-b880-72af96e2a75a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fb58643 2021-05-28T01:00:43.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6642be3b-2b91-403e-b880-72af96e2a75a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:43.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6642be3b-2b91-403e-b880-72af96e2a75a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:46.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a506f6-c7ef-4f17-a693-1505742532e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:46.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a506f6-c7ef-4f17-a693-1505742532e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c5cb03a 2021-05-28T01:00:46.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a506f6-c7ef-4f17-a693-1505742532e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:46.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a506f6-c7ef-4f17-a693-1505742532e0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:50.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=044e0e16-e94f-41ed-9da2-b252cb97defb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:50.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=044e0e16-e94f-41ed-9da2-b252cb97defb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@404f95f6 2021-05-28T01:00:50.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=044e0e16-e94f-41ed-9da2-b252cb97defb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:50.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=044e0e16-e94f-41ed-9da2-b252cb97defb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:00:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:00:52.124Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:00:54.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a5a0352-2402-48c8-a90a-6921a1d441ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:54.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a5a0352-2402-48c8-a90a-6921a1d441ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d476cab 2021-05-28T01:00:54.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a5a0352-2402-48c8-a90a-6921a1d441ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:54.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a5a0352-2402-48c8-a90a-6921a1d441ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:00:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:00:57.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d67909c-059b-4310-922a-aa94c1e6d8da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:57.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d67909c-059b-4310-922a-aa94c1e6d8da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e9366de 2021-05-28T01:00:57.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d67909c-059b-4310-922a-aa94c1e6d8da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:57.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d67909c-059b-4310-922a-aa94c1e6d8da PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:00:57.989Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6d67909c-059b-4310-922a-aa94c1e6d8da ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:00:57.989Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=6d67909c-059b-4310-922a-aa94c1e6d8da ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:00:58.868Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:00:59.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf5fca84-b74b-43bf-b852-984eb2bed538 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:00:59.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf5fca84-b74b-43bf-b852-984eb2bed538 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7777c256 2021-05-28T01:00:59.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf5fca84-b74b-43bf-b852-984eb2bed538 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:00:59.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf5fca84-b74b-43bf-b852-984eb2bed538 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:02.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a143592a-771f-4efb-bf43-79c6815ad1b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:02.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a143592a-771f-4efb-bf43-79c6815ad1b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b2b196b 2021-05-28T01:01:02.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a143592a-771f-4efb-bf43-79c6815ad1b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:02.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a143592a-771f-4efb-bf43-79c6815ad1b1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:02.860Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a143592a-771f-4efb-bf43-79c6815ad1b1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:01:02.860Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a143592a-771f-4efb-bf43-79c6815ad1b1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:01:05.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a08455eb-b3d6-49d5-9aec-14753a999494 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:05.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a08455eb-b3d6-49d5-9aec-14753a999494 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6daf0691 2021-05-28T01:01:05.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a08455eb-b3d6-49d5-9aec-14753a999494 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:05.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a08455eb-b3d6-49d5-9aec-14753a999494 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:05.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:01:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:01:08.155Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node012o2yig8onq2bj6hddazw0kte186 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xx8dyeus312416pnz9qfid3xk222 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01npyyq4006qm117pe85bp6dby2204 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01q0s9rep99phigqkgyr7zyy14206 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node016i0heu3dmi5x1xagom4j5gzp9241 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ft0tjyt5hcae1cvbxc2bgbizi231 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node064dx084t508v14x2bexd5of38228 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01emxbe5xi1rrl1l0ist1xw2bkw236 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01930njtqzf7ldupd9lvc9uzju188 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0evurq6afpko61qiufg6x8bstn219 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node048jfv1p5vmxo155knwra4znz2223 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0y6udqcj3eipw5xfgj37uqeeu199 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0nlmr17llf96cvdrdcdjtdufu191 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node018hxog9om3m1dee96lxufrqju248 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0syihu94xy7pd1jmkbn8uwpd4q246 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node076f5nllwali5qf4tovcvw0rj197 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xwr55ptn22nh1qvbbwc70gjux198 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node014aoa466f74ff78yupb20fgkf217 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node017yz8uq93el7b1whdnhkyaa3un244 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0cpq9olvbdomcspvrlygnj88g218 2021-05-28T01:01:08.156Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yexzl6pr6i9r1ttcpmkrj3nwc208 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node016cdhd8z8vr0nfa1l93fnvdgr212 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0uuvmz5nlf9khaq1figrecn93201 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node035fztvqtdowfwy2nsg889yod193 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01p8pkaq995my9uqymlm1vupzt194 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ac71pkppar0tyh0bfe9ewo0t211 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011x2ls596ueljq6p5q7ug5fxk205 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xd9osxo15t9k1qgvv3brst7yl220 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01nw8dl5cnuotg16jdh6qgrbkm5183 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0fm8oa9yjsz4c1xxeosh43ce71240 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ntx8n34lbi3k1q40tud7w04l4247 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011ieywu0fxjgr1awra41ofpzsq214 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01v6nprd6bf2t01ihyljql8obdl251 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0kef89qiv3obf1n4iwmh90296w202 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01f62ak2dvnxxg1vr2i8gqjv1o3235 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0443bzirslnonc4c12zark32o242 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01nnmxeifugysejxh27z1htuo3232 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gn64x3lds6ku16ym1lhfy3sd5185 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node019k1xuoakqiju1tqpgvmrintvi227 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gsrvwduao6721wor9u68khk5y250 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0o3ako8qooyie80bigzijern5203 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0nm1cnvkxvu13x2jcvasekd1189 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0s9p1sbeh59nm171gsh4qe9rq5239 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0um3sxsa07wrq1guwkpx8fyrj7245 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node05whvxnchgg521b86oani252eu209 2021-05-28T01:01:08.157Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01bod0l1zjtnky1l3uqjh27mxhq243 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0nxr5vwi6quhodmbj7u2hvyxa229 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0nf3w4u1hzbcsi7oy78genq3f234 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01vwt4xt6bkxkx13gyrwfk5awiy190 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xgr51y4qsvfi1fxshrhaf8r9d238 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kjdvedgp6jw8cfr0xk5rmeqo237 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0147aeoenywt1hly51t1nzl5wi226 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01tz0oqgscgtppuhygkbf0w7rx216 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node017pznhy4recmj1lynfkp5fs0on233 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ndfc8z928fhhl1cpqh7f613v213 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0d72rwr3gv9vg1jlmahfmwkg0q230 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09zwq9b28yojsg6a2ktxtt1ls224 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node014weo28bdmu3pmgk3iwg47tug195 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0gm8e90ugc0vnlz2nsxhspq1a184 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01i06lz5g3f7vd9x7cwa3mrob3249 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01b9i6ooka3lwkthc7w8stavtc221 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0lzc680jyudn5dmmszes99uc6215 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0qdtvgxt4dgf0ql7h7zn1vvrg207 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010i6aj46rzo403hj0kby62ez7252 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01vza4wgi0fw8u12ldoyehrsrzu187 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01j62nfhmn2zni1s4fe5wzc803g225 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010pvnp1tx5bc1n6knsx474rpc192 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0trgfcf6pgtvl11ai9k9mllmb5200 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01iv0ole6ppdwm1q6qupfa1n54x196 2021-05-28T01:01:08.158Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06isodd193z2t1fg9pto5adcfo210 2021-05-28T01:01:08.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44947bd5-22ff-4a7e-8e95-df02a79c4a03 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:08.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44947bd5-22ff-4a7e-8e95-df02a79c4a03 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@385f286c 2021-05-28T01:01:08.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44947bd5-22ff-4a7e-8e95-df02a79c4a03 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:08.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44947bd5-22ff-4a7e-8e95-df02a79c4a03 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3ad2da9-b9d0-4c49-ac79-39c677259379 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3ad2da9-b9d0-4c49-ac79-39c677259379 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71732843 2021-05-28T01:01:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3ad2da9-b9d0-4c49-ac79-39c677259379 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:10.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3ad2da9-b9d0-4c49-ac79-39c677259379 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:14.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a3db88-6fe7-4d9b-b018-bae94ea341a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:14.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a3db88-6fe7-4d9b-b018-bae94ea341a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62809747 2021-05-28T01:01:14.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a3db88-6fe7-4d9b-b018-bae94ea341a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:14.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40a3db88-6fe7-4d9b-b018-bae94ea341a8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:16.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71748362-0072-4de8-aaac-11e871238f61 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:16.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71748362-0072-4de8-aaac-11e871238f61 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@778e9e08 2021-05-28T01:01:16.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71748362-0072-4de8-aaac-11e871238f61 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:16.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71748362-0072-4de8-aaac-11e871238f61 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:19.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4697c4f1-89dc-4ea4-be0a-3de3b4a3eb4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:19.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4697c4f1-89dc-4ea4-be0a-3de3b4a3eb4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36d3c4dd 2021-05-28T01:01:19.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4697c4f1-89dc-4ea4-be0a-3de3b4a3eb4d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:19.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4697c4f1-89dc-4ea4-be0a-3de3b4a3eb4d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:01:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:01:22.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e20482-a9ae-47c9-9e38-7678bafcf980 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:22.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e20482-a9ae-47c9-9e38-7678bafcf980 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e8ac374 2021-05-28T01:01:22.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e20482-a9ae-47c9-9e38-7678bafcf980 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:22.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82e20482-a9ae-47c9-9e38-7678bafcf980 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd2fe00e-0c49-497a-b76e-2976add4abb4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd2fe00e-0c49-497a-b76e-2976add4abb4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38565d4d 2021-05-28T01:01:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd2fe00e-0c49-497a-b76e-2976add4abb4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:26.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd2fe00e-0c49-497a-b76e-2976add4abb4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:29.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df887a3-1623-41b0-9d3c-81b5ad8820c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:29.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df887a3-1623-41b0-9d3c-81b5ad8820c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b2bea91 2021-05-28T01:01:29.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df887a3-1623-41b0-9d3c-81b5ad8820c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:29.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df887a3-1623-41b0-9d3c-81b5ad8820c5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:31.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d233a3-acc1-4818-ba4b-cffb9db199f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:31.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d233a3-acc1-4818-ba4b-cffb9db199f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13762405 2021-05-28T01:01:31.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d233a3-acc1-4818-ba4b-cffb9db199f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:31.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d233a3-acc1-4818-ba4b-cffb9db199f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:34.910Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d230b719-9ec0-416d-9fee-ca9b06472946 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:34.910Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d230b719-9ec0-416d-9fee-ca9b06472946 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1befd47c 2021-05-28T01:01:34.910Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d230b719-9ec0-416d-9fee-ca9b06472946 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:34.910Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d230b719-9ec0-416d-9fee-ca9b06472946 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:01:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:01:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:01:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:01:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:01:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b43c1634-6b47-4a75-b5f5-a38cf970ecb8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b43c1634-6b47-4a75-b5f5-a38cf970ecb8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11595e79 2021-05-28T01:01:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b43c1634-6b47-4a75-b5f5-a38cf970ecb8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b43c1634-6b47-4a75-b5f5-a38cf970ecb8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:40.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bcbeac-f5e8-4e1d-9840-36e2da00e085 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:40.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bcbeac-f5e8-4e1d-9840-36e2da00e085 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75c241a1 2021-05-28T01:01:40.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bcbeac-f5e8-4e1d-9840-36e2da00e085 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:40.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bcbeac-f5e8-4e1d-9840-36e2da00e085 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:43.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464aa7e7-a2aa-4792-a6f9-9fd64b62aaab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:43.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464aa7e7-a2aa-4792-a6f9-9fd64b62aaab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57360510 2021-05-28T01:01:43.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464aa7e7-a2aa-4792-a6f9-9fd64b62aaab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:43.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464aa7e7-a2aa-4792-a6f9-9fd64b62aaab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8586ffe-520e-46f9-9d68-354d36cc7871 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8586ffe-520e-46f9-9d68-354d36cc7871 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@321f77be 2021-05-28T01:01:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8586ffe-520e-46f9-9d68-354d36cc7871 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8586ffe-520e-46f9-9d68-354d36cc7871 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ab942b0-1f78-4a0d-9170-2a9f4f069b9e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ab942b0-1f78-4a0d-9170-2a9f4f069b9e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@696e4d12 2021-05-28T01:01:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ab942b0-1f78-4a0d-9170-2a9f4f069b9e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:49.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ab942b0-1f78-4a0d-9170-2a9f4f069b9e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:01:50.517Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:01:51.701Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:01:52.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47af7f48-7c9a-401f-91bf-44fa9cfcac3d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:52.787Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47af7f48-7c9a-401f-91bf-44fa9cfcac3d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e46d6d7 2021-05-28T01:01:52.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47af7f48-7c9a-401f-91bf-44fa9cfcac3d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:52.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47af7f48-7c9a-401f-91bf-44fa9cfcac3d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:55.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dd59fa-4410-414e-9dde-5a348fb7209c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:55.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dd59fa-4410-414e-9dde-5a348fb7209c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69bf4c8a 2021-05-28T01:01:55.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dd59fa-4410-414e-9dde-5a348fb7209c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:55.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dd59fa-4410-414e-9dde-5a348fb7209c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:01:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d5c1808-f119-4961-848c-de69a129c474 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:01:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d5c1808-f119-4961-848c-de69a129c474 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20d47484 2021-05-28T01:01:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d5c1808-f119-4961-848c-de69a129c474 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:01:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d5c1808-f119-4961-848c-de69a129c474 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:01.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09eb21e-84f7-4b49-9170-8847bb2c3cc3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:01.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09eb21e-84f7-4b49-9170-8847bb2c3cc3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b49ab51 2021-05-28T01:02:01.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09eb21e-84f7-4b49-9170-8847bb2c3cc3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:01.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09eb21e-84f7-4b49-9170-8847bb2c3cc3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1e31dbd-676e-4bc3-a402-5ca897b8e250 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1e31dbd-676e-4bc3-a402-5ca897b8e250 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3812281 2021-05-28T01:02:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1e31dbd-676e-4bc3-a402-5ca897b8e250 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:04.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1e31dbd-676e-4bc3-a402-5ca897b8e250 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:02:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:02:07.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1218cba-8fae-4809-af0c-1ff69c2922f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:07.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1218cba-8fae-4809-af0c-1ff69c2922f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1049f567 2021-05-28T01:02:07.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1218cba-8fae-4809-af0c-1ff69c2922f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:07.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1218cba-8fae-4809-af0c-1ff69c2922f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:11.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081e6751-7e2e-4254-b8b5-01e5fee06b39 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:11.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081e6751-7e2e-4254-b8b5-01e5fee06b39 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13bac3be 2021-05-28T01:02:11.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081e6751-7e2e-4254-b8b5-01e5fee06b39 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:11.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081e6751-7e2e-4254-b8b5-01e5fee06b39 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:13.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d149d808-4dfe-4984-8634-a41bbe89eacb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:13.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d149d808-4dfe-4984-8634-a41bbe89eacb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@655ff1bb 2021-05-28T01:02:13.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d149d808-4dfe-4984-8634-a41bbe89eacb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:13.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d149d808-4dfe-4984-8634-a41bbe89eacb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:16.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3a215e-9bf3-4a98-a46b-ec86402b75d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:16.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3a215e-9bf3-4a98-a46b-ec86402b75d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f4052bb 2021-05-28T01:02:16.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3a215e-9bf3-4a98-a46b-ec86402b75d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:16.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3a215e-9bf3-4a98-a46b-ec86402b75d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=248b3219-a0ab-4e08-b84c-6dbdc33d118d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=248b3219-a0ab-4e08-b84c-6dbdc33d118d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@307d70f9 2021-05-28T01:02:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=248b3219-a0ab-4e08-b84c-6dbdc33d118d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=248b3219-a0ab-4e08-b84c-6dbdc33d118d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:02:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:02:23.362Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=50e5629d-2c25-4459-9914-0bd2f429905f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:23.361622Z InvocationID=dbf8ad27-15cd-4320-8487-940819577e7c LogTimestamp=2021-05-28T01:02:23.361776Z Entering 2021-05-28T01:02:23.362Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=50e5629d-2c25-4459-9914-0bd2f429905f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:23.361622Z InvocationID=dbf8ad27-15cd-4320-8487-940819577e7c LogTimestamp=2021-05-28T01:02:23.361776Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T01:02:23.386Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=24 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T01:02:23.362Z InvocationID=e3d67b7f-dacc-4c5f-82a1-d085696f7212 LogTimestamp=2021-05-28T01:02:23.386285Z ResponseDesc=OK Exiting. 2021-05-28T01:02:23.386Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T01:02:23.564Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:23.564308Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:23.564Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:23.564308Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:23.564Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-05-28T01:02:23.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afed6620-91fc-45ca-8ede-459e7be7b2de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:23.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afed6620-91fc-45ca-8ede-459e7be7b2de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@751ff27b 2021-05-28T01:02:23.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afed6620-91fc-45ca-8ede-459e7be7b2de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:23.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afed6620-91fc-45ca-8ede-459e7be7b2de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:23.669Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1578f156-a92c-4c3b-a962-2f372e69365a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:23.669286Z InvocationID=97ca55fe-1755-4eb6-ab92-46b9cde50e7c LogTimestamp=2021-05-28T01:02:23.669328Z Entering 2021-05-28T01:02:23.669Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1578f156-a92c-4c3b-a962-2f372e69365a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:23.669286Z InvocationID=97ca55fe-1755-4eb6-ab92-46b9cde50e7c LogTimestamp=2021-05-28T01:02:23.669328Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T01:02:23.868Z [qtp580718781-19] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T01:02:23.669Z InvocationID=4b51cda0-9cc9-4a7b-83ff-f24092e725e1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@73bd41f3', 'User [firstName=demo, lastName=demo, userId=demo, email=demo@openecomp.org, role=ADMIN, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 197ms 2021-05-28T01:02:23.869Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'demo' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=200 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T01:02:23.669Z InvocationID=4b51cda0-9cc9-4a7b-83ff-f24092e725e1 ErrorCode=409 LogTimestamp=2021-05-28T01:02:23.869028Z ResponseDesc=Conflict Exiting. 2021-05-28T01:02:23.869Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T01:02:23.972Z [qtp580718781-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:23.971941Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:23.972Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:23.971941Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:23.972Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-05-28T01:02:24.311Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=66f25e29-24de-42be-9c0c-a2cb51316f39 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:24.311096Z InvocationID=c8bb73b5-4681-4560-903b-c1d3acc1f4bd LogTimestamp=2021-05-28T01:02:24.311128Z Entering 2021-05-28T01:02:24.311Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=66f25e29-24de-42be-9c0c-a2cb51316f39 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:24.311096Z InvocationID=c8bb73b5-4681-4560-903b-c1d3acc1f4bd LogTimestamp=2021-05-28T01:02:24.311128Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T01:02:24.660Z [qtp580718781-12] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T01:02:24.311Z InvocationID=a461c208-0acf-451e-bc0b-7ad39185ac17 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@73bd41f3', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 348ms 2021-05-28T01:02:24.661Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=349 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T01:02:24.311Z InvocationID=a461c208-0acf-451e-bc0b-7ad39185ac17 ErrorCode=400 LogTimestamp=2021-05-28T01:02:24.660880Z ResponseDesc=Bad Request Exiting. 2021-05-28T01:02:24.661Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T01:02:25.071Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=70248 PartnerName=kube-probe/1.18 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:01:14.823Z InvocationID=d775537f-ec66-44d6-9d8d-2475402cb532 ErrorCode=500 LogTimestamp=2021-05-28T01:02:25.071814Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:25.072Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-05-28T01:02:25.458Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=411af4f9-eb26-492b-95d5-4989383b17a4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:25.458016Z InvocationID=a650e310-2783-4635-a38d-0249612690d6 LogTimestamp=2021-05-28T01:02:25.458066Z Entering 2021-05-28T01:02:25.458Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=411af4f9-eb26-492b-95d5-4989383b17a4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:25.458016Z InvocationID=a650e310-2783-4635-a38d-0249612690d6 LogTimestamp=2021-05-28T01:02:25.458066Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T01:02:25.764Z [qtp580718781-11] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T01:02:25.458Z InvocationID=e9a5cc6a-650c-419e-97f4-a1964c7ecd0b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@73bd41f3', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[165] in 304ms 2021-05-28T01:02:25.765Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=306 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-05-28T01:02:25.458Z InvocationID=e9a5cc6a-650c-419e-97f4-a1964c7ecd0b ErrorCode=400 LogTimestamp=2021-05-28T01:02:25.764928Z ResponseDesc=Bad Request Exiting. 2021-05-28T01:02:25.765Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-05-28T01:02:25.875Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:25.875152Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:25.875Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:25.875152Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:25.875Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-05-28T01:02:26.064Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1764eb27-9594-4f24-915e-29886fe47c2f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:26.063850Z InvocationID=bbf86b9a-e3e6-4be5-81ff-d7bceb189d39 LogTimestamp=2021-05-28T01:02:26.063884Z Entering 2021-05-28T01:02:26.064Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1764eb27-9594-4f24-915e-29886fe47c2f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:26.063850Z InvocationID=bbf86b9a-e3e6-4be5-81ff-d7bceb189d39 LogTimestamp=2021-05-28T01:02:26.063884Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T01:02:26.161Z [qtp580718781-16] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T01:02:26.064Z InvocationID=270ebcdb-7b63-47c7-8937-10621cea5726 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@73bd41f3', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 96ms 2021-05-28T01:02:26.162Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T01:02:26.064Z InvocationID=270ebcdb-7b63-47c7-8937-10621cea5726 ErrorCode=409 LogTimestamp=2021-05-28T01:02:26.162271Z ResponseDesc=Conflict Exiting. 2021-05-28T01:02:26.163Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T01:02:26.258Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:26.258331Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:26.258Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:26.258331Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:26.258Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-05-28T01:02:26.268Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=532a4d40-c12e-4230-8927-29938995b9d3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:26.268181Z InvocationID=c75e8bdf-867d-45c3-8242-bbce5b133a7d LogTimestamp=2021-05-28T01:02:26.268221Z Entering 2021-05-28T01:02:26.268Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=532a4d40-c12e-4230-8927-29938995b9d3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:26.268181Z InvocationID=c75e8bdf-867d-45c3-8242-bbce5b133a7d LogTimestamp=2021-05-28T01:02:26.268221Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T01:02:26.370Z [qtp580718781-15] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T01:02:26.268Z InvocationID=f3675118-d797-41ca-ad53-6fd2ab452342 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@73bd41f3', 'User [firstName=Joni, lastName=Mitchell, userId=jm0007, email=jm0007@openecomp.org, role=TESTER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 101ms 2021-05-28T01:02:26.371Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'jm0007' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=102 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T01:02:26.268Z InvocationID=f3675118-d797-41ca-ad53-6fd2ab452342 ErrorCode=409 LogTimestamp=2021-05-28T01:02:26.370961Z ResponseDesc=Conflict Exiting. 2021-05-28T01:02:26.371Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T01:02:26.466Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:26.466420Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:26.466Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:26.466420Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:26.466Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-05-28T01:02:26.563Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0fa37831-144e-4c57-a64e-8005e97fc034 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:26.562945Z InvocationID=10a91918-ad07-4862-838d-fd4eea387a01 LogTimestamp=2021-05-28T01:02:26.562992Z Entering 2021-05-28T01:02:26.563Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0fa37831-144e-4c57-a64e-8005e97fc034 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/user ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:26.562945Z InvocationID=10a91918-ad07-4862-838d-fd4eea387a01 LogTimestamp=2021-05-28T01:02:26.562992Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/user HTTP/1.1 2021-05-28T01:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fc1b583-0e27-495c-848d-186700b0f91b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fc1b583-0e27-495c-848d-186700b0f91b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fd7074a 2021-05-28T01:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fc1b583-0e27-495c-848d-186700b0f91b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:26.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fc1b583-0e27-495c-848d-186700b0f91b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:27.560Z [qtp580718781-18] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/user EntryTimestamp=2021-05-28T01:02:26.563Z InvocationID=867b1f13-843f-41eb-9f0a-413b6a318bc9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@73bd41f3', 'User [firstName=Carlos, lastName=Santana, userId=cs0008, email=cs0008r@openecomp.org, role=DESIGNER, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1271] in 996ms 2021-05-28T01:02:27.560Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4006: Error: User with 'cs0008' ID already exists." ResponseCode=409 RequestID=null ElapsedTime=997 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-05-28T01:02:26.563Z InvocationID=867b1f13-843f-41eb-9f0a-413b6a318bc9 ErrorCode=409 LogTimestamp=2021-05-28T01:02:27.560772Z ResponseDesc=Conflict Exiting. 2021-05-28T01:02:27.561Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-05-28T01:02:28.269Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f042713a-2b7a-4a2a-a4a6-65a526e1c9d8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:28.269523Z InvocationID=de01ad27-533c-4e0e-88eb-e2b50acf45e2 LogTimestamp=2021-05-28T01:02:28.269557Z Entering 2021-05-28T01:02:28.269Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f042713a-2b7a-4a2a-a4a6-65a526e1c9d8 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:28.269523Z InvocationID=de01ad27-533c-4e0e-88eb-e2b50acf45e2 LogTimestamp=2021-05-28T01:02:28.269557Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T01:02:28.775Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=506 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T01:02:28.269Z InvocationID=0172dd08-578f-40d2-972c-a15c45eaaaca LogTimestamp=2021-05-28T01:02:28.775728Z ResponseDesc=OK Exiting. 2021-05-28T01:02:28.776Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T01:02:28.870Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:28.870285Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:28.870Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:28.870285Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:28.870Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-05-28T01:02:28.969Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=23e82d02-0c28-4510-8b1f-70b54530c0f3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:28.969129Z InvocationID=075bd5be-c3ea-4b0a-a4fe-7ea4bfcb1152 LogTimestamp=2021-05-28T01:02:28.969172Z Entering 2021-05-28T01:02:28.969Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=23e82d02-0c28-4510-8b1f-70b54530c0f3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:28.969129Z InvocationID=075bd5be-c3ea-4b0a-a4fe-7ea4bfcb1152 LogTimestamp=2021-05-28T01:02:28.969172Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:28.978Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:28.969Z InvocationID=b448d5bd-4db0-431a-94c1-43329eef15f3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id aai 2021-05-28T01:02:29.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eee0ba2-e296-4c62-b0b5-60a5c2303b72 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:29.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eee0ba2-e296-4c62-b0b5-60a5c2303b72 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26f5581 2021-05-28T01:02:29.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eee0ba2-e296-4c62-b0b5-60a5c2303b72 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:29.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8eee0ba2-e296-4c62-b0b5-60a5c2303b72 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:32.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c61dcfc6-8e85-4dc0-a538-f00952ed54e8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:32.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c61dcfc6-8e85-4dc0-a538-f00952ed54e8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59ce178d 2021-05-28T01:02:32.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c61dcfc6-8e85-4dc0-a538-f00952ed54e8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:32.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c61dcfc6-8e85-4dc0-a538-f00952ed54e8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:34.574Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=5605 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:28.969Z InvocationID=b448d5bd-4db0-431a-94c1-43329eef15f3 LogTimestamp=2021-05-28T01:02:34.574183Z ResponseDesc=Created Exiting. 2021-05-28T01:02:34.574Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:34.768Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:34.768859Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:34.769Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:34.768859Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:34.769Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-05-28T01:02:34.875Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0741b960-faad-4223-a233-c33c11dce7ea InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:34.875491Z InvocationID=23cd0cba-3e8e-41e3-b6e9-d347425de2c4 LogTimestamp=2021-05-28T01:02:34.875542Z Entering 2021-05-28T01:02:34.875Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0741b960-faad-4223-a233-c33c11dce7ea InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:34.875491Z InvocationID=23cd0cba-3e8e-41e3-b6e9-d347425de2c4 LogTimestamp=2021-05-28T01:02:34.875542Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:34.890Z [qtp580718781-96] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:34.875Z InvocationID=e23a018f-52ce-45e2-a27b-df817360f7c9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id appc 2021-05-28T01:02:34.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f0b4b7-d46e-4467-a984-fde5bbc6fff5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f0b4b7-d46e-4467-a984-fde5bbc6fff5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53c8bae5 2021-05-28T01:02:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f0b4b7-d46e-4467-a984-fde5bbc6fff5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:34.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8f0b4b7-d46e-4467-a984-fde5bbc6fff5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:02:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:02:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:02:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:02:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:02:37.164Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2288 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:34.875Z InvocationID=e23a018f-52ce-45e2-a27b-df817360f7c9 LogTimestamp=2021-05-28T01:02:37.163894Z ResponseDesc=Created Exiting. 2021-05-28T01:02:37.164Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:37.268Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:37.268845Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:37.268Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:37.268845Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:37.269Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-05-28T01:02:37.375Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=068b349f-d0ff-4c13-a195-4d875ee84281 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:37.375132Z InvocationID=34292336-09f8-434c-9ec8-9e6579941a9f LogTimestamp=2021-05-28T01:02:37.375167Z Entering 2021-05-28T01:02:37.375Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=068b349f-d0ff-4c13-a195-4d875ee84281 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:37.375132Z InvocationID=34292336-09f8-434c-9ec8-9e6579941a9f LogTimestamp=2021-05-28T01:02:37.375167Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:37.556Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:37.375Z InvocationID=a91f718e-66cd-4e28-8962-6bde4ff2d86a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id dcae 2021-05-28T01:02:37.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2d03da-dbbc-4fea-8607-1345b3efd821 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:37.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2d03da-dbbc-4fea-8607-1345b3efd821 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f21a81 2021-05-28T01:02:37.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2d03da-dbbc-4fea-8607-1345b3efd821 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:37.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2d03da-dbbc-4fea-8607-1345b3efd821 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:41.658Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4283 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:37.375Z InvocationID=a91f718e-66cd-4e28-8962-6bde4ff2d86a LogTimestamp=2021-05-28T01:02:41.658752Z ResponseDesc=Created Exiting. 2021-05-28T01:02:41.659Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:41.759Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:41.759754Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:41.759Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:41.759754Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:41.760Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-05-28T01:02:41.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32389b93-4d41-42e4-9ccd-ee44f652b8d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:41.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32389b93-4d41-42e4-9ccd-ee44f652b8d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c4984c3 2021-05-28T01:02:41.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32389b93-4d41-42e4-9ccd-ee44f652b8d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:41.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32389b93-4d41-42e4-9ccd-ee44f652b8d1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:41.859Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6c0b7fd1-db28-40c4-b621-b67f39fa6912 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:41.859113Z InvocationID=de97e8e7-25c5-4cad-b6ec-1b27edd5bb23 LogTimestamp=2021-05-28T01:02:41.859189Z Entering 2021-05-28T01:02:41.859Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6c0b7fd1-db28-40c4-b621-b67f39fa6912 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:41.859113Z InvocationID=de97e8e7-25c5-4cad-b6ec-1b27edd5bb23 LogTimestamp=2021-05-28T01:02:41.859189Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:41.870Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:41.859Z InvocationID=2b8bcb33-bbf2-4293-965e-da5c6164bb9f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id mso 2021-05-28T01:02:43.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ad227ff-7ac4-4291-969c-1394f2c875b5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:43.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ad227ff-7ac4-4291-969c-1394f2c875b5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@551286b8 2021-05-28T01:02:43.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ad227ff-7ac4-4291-969c-1394f2c875b5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:43.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ad227ff-7ac4-4291-969c-1394f2c875b5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:44.166Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2306 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:41.859Z InvocationID=2b8bcb33-bbf2-4293-965e-da5c6164bb9f LogTimestamp=2021-05-28T01:02:44.165884Z ResponseDesc=Created Exiting. 2021-05-28T01:02:44.166Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:44.459Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:44.459089Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:44.459Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:44.459089Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:44.459Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-05-28T01:02:44.583Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=763bec28-e72d-4c01-8544-f2adeabe615e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:44.583323Z InvocationID=36fc1c98-cc8c-475a-a31e-63bd316da762 LogTimestamp=2021-05-28T01:02:44.583385Z Entering 2021-05-28T01:02:44.583Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=763bec28-e72d-4c01-8544-f2adeabe615e InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:44.583323Z InvocationID=36fc1c98-cc8c-475a-a31e-63bd316da762 LogTimestamp=2021-05-28T01:02:44.583385Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:44.764Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:44.583Z InvocationID=fad09904-f05f-40f6-b9f7-add8aaf44945 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id sdnc 2021-05-28T01:02:46.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b86937-e58f-4af5-9987-bf5db1616556 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:46.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b86937-e58f-4af5-9987-bf5db1616556 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@439b016d 2021-05-28T01:02:46.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b86937-e58f-4af5-9987-bf5db1616556 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:46.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66b86937-e58f-4af5-9987-bf5db1616556 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:47.280Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2697 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:44.583Z InvocationID=fad09904-f05f-40f6-b9f7-add8aaf44945 LogTimestamp=2021-05-28T01:02:47.280631Z ResponseDesc=Created Exiting. 2021-05-28T01:02:47.281Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:47.371Z [qtp580718781-98] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:47.371422Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:47.371Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:47.371422Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:47.371Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-05-28T01:02:47.468Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=471474ea-4b8a-4ad4-a17a-eac0070d326d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:47.468092Z InvocationID=8468f52a-1d15-4177-8201-8e71b0b1b029 LogTimestamp=2021-05-28T01:02:47.468144Z Entering 2021-05-28T01:02:47.468Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=471474ea-4b8a-4ad4-a17a-eac0070d326d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:47.468092Z InvocationID=8468f52a-1d15-4177-8201-8e71b0b1b029 LogTimestamp=2021-05-28T01:02:47.468144Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:47.491Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:47.468Z InvocationID=f139a409-c4e0-435c-a92e-2a0594408751 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vid 2021-05-28T01:02:49.075Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1607 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:47.468Z InvocationID=f139a409-c4e0-435c-a92e-2a0594408751 LogTimestamp=2021-05-28T01:02:49.075572Z ResponseDesc=Created Exiting. 2021-05-28T01:02:49.076Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:49.269Z [qtp580718781-19] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:49.268961Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:49.269Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:49.268961Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:49.269Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-05-28T01:02:49.277Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0ba97572-0552-4c80-94ef-385d03228050 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:49.277501Z InvocationID=3448eb14-b134-4b66-8f40-c8520e725a2a LogTimestamp=2021-05-28T01:02:49.277534Z Entering 2021-05-28T01:02:49.277Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0ba97572-0552-4c80-94ef-385d03228050 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:49.277501Z InvocationID=3448eb14-b134-4b66-8f40-c8520e725a2a LogTimestamp=2021-05-28T01:02:49.277534Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:49.468Z [qtp580718781-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:49.277Z InvocationID=91a84919-495d-4561-817e-18461492db06 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cognita 2021-05-28T01:02:49.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdc50aa-2c8e-4941-92e4-268f48fb0502 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:49.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdc50aa-2c8e-4941-92e4-268f48fb0502 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57292d8d 2021-05-28T01:02:49.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdc50aa-2c8e-4941-92e4-268f48fb0502 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:49.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfdc50aa-2c8e-4941-92e4-268f48fb0502 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:02:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:02:51.362Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2085 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:49.277Z InvocationID=91a84919-495d-4561-817e-18461492db06 LogTimestamp=2021-05-28T01:02:51.362018Z ResponseDesc=Created Exiting. 2021-05-28T01:02:51.362Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:51.471Z [qtp580718781-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:51.471313Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:51.471Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:51.471313Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:51.471Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-05-28T01:02:51.686Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=569c0865-216d-488a-acd1-0a9687e77f99 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:51.686665Z InvocationID=71569ade-54ab-454c-90ac-62ed1efc9305 LogTimestamp=2021-05-28T01:02:51.686706Z Entering 2021-05-28T01:02:51.686Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=569c0865-216d-488a-acd1-0a9687e77f99 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:51.686665Z InvocationID=71569ade-54ab-454c-90ac-62ed1efc9305 LogTimestamp=2021-05-28T01:02:51.686706Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:51.856Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:51.687Z InvocationID=79c4079c-13ac-4df8-b64f-c48d12b84d51 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id clamp 2021-05-28T01:02:52.127Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:02:52.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d20a6df-2c48-4d2e-a359-bcb15608ed5a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:52.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d20a6df-2c48-4d2e-a359-bcb15608ed5a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6776d437 2021-05-28T01:02:52.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d20a6df-2c48-4d2e-a359-bcb15608ed5a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:52.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d20a6df-2c48-4d2e-a359-bcb15608ed5a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:53.566Z [qtp580718781-16] WARN o.j.d.l.c.ConsistentKeyLocker RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:51.687Z InvocationID=79c4079c-13ac-4df8-b64f-c48d12b84d51 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk Lock write succeeded but took too long: duration PT0.504344S exceeded limit PT0.5S 2021-05-28T01:02:54.371Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2684 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:51.687Z InvocationID=79c4079c-13ac-4df8-b64f-c48d12b84d51 LogTimestamp=2021-05-28T01:02:54.371031Z ResponseDesc=Created Exiting. 2021-05-28T01:02:54.371Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:54.479Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:54.479349Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:54.479Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:54.479349Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:54.479Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-05-28T01:02:54.759Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7169d629-334e-43d8-a892-3c4b34378fdb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:54.759601Z InvocationID=dbd67131-b101-4082-87d8-766bf24d5866 LogTimestamp=2021-05-28T01:02:54.759663Z Entering 2021-05-28T01:02:54.759Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7169d629-334e-43d8-a892-3c4b34378fdb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:54.759601Z InvocationID=dbd67131-b101-4082-87d8-766bf24d5866 LogTimestamp=2021-05-28T01:02:54.759663Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:54.880Z [qtp580718781-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:54.760Z InvocationID=71a54d85-bed1-4c8a-8eb6-561560bc1e33 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id vfc 2021-05-28T01:02:55.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23b6d98e-8b5e-4055-b0a8-38376359e79e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:55.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23b6d98e-8b5e-4055-b0a8-38376359e79e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2690b927 2021-05-28T01:02:55.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23b6d98e-8b5e-4055-b0a8-38376359e79e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:55.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23b6d98e-8b5e-4055-b0a8-38376359e79e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:56.679Z [qtp580718781-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1918 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:54.760Z InvocationID=71a54d85-bed1-4c8a-8eb6-561560bc1e33 LogTimestamp=2021-05-28T01:02:56.678964Z ResponseDesc=Created Exiting. 2021-05-28T01:02:56.679Z [qtp580718781-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:56.772Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:56.772573Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:56.772Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:56.772573Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:56.773Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-05-28T01:02:56.972Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a2a22e19-d408-4781-94aa-1d6ed3cc883f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:56.971928Z InvocationID=db77510e-f4a0-4e0b-80c7-6b8e031f306e LogTimestamp=2021-05-28T01:02:56.971976Z Entering 2021-05-28T01:02:56.972Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a2a22e19-d408-4781-94aa-1d6ed3cc883f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:56.971928Z InvocationID=db77510e-f4a0-4e0b-80c7-6b8e031f306e LogTimestamp=2021-05-28T01:02:56.971976Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:56.985Z [qtp580718781-98] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:56.972Z InvocationID=deb751d8-b008-4353-8def-82efc082b1d3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id workflow 2021-05-28T01:02:58.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00797541-cff8-4d0f-9bda-c790ff9ef814 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:02:58.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00797541-cff8-4d0f-9bda-c790ff9ef814 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d1b54f1 2021-05-28T01:02:58.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00797541-cff8-4d0f-9bda-c790ff9ef814 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:02:58.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00797541-cff8-4d0f-9bda-c790ff9ef814 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:02:58.573Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1601 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:56.972Z InvocationID=deb751d8-b008-4353-8def-82efc082b1d3 LogTimestamp=2021-05-28T01:02:58.573190Z ResponseDesc=Created Exiting. 2021-05-28T01:02:58.573Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:02:58.670Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:58.670511Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:02:58.670Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:02:58.670511Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:02:58.671Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-05-28T01:02:58.679Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=21f223f1-1667-4fa4-9516-3612222ab7bf InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:58.679557Z InvocationID=8f9223dc-33e9-4ebf-b969-903a19dba88f LogTimestamp=2021-05-28T01:02:58.679601Z Entering 2021-05-28T01:02:58.679Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=21f223f1-1667-4fa4-9516-3612222ab7bf InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:02:58.679557Z InvocationID=8f9223dc-33e9-4ebf-b969-903a19dba88f LogTimestamp=2021-05-28T01:02:58.679601Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:02:58.762Z [qtp580718781-19] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:58.680Z InvocationID=d51570cf-2960-42b4-8bc6-82baf8f71b53 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id policy 2021-05-28T01:03:00.577Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1897 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:02:58.680Z InvocationID=d51570cf-2960-42b4-8bc6-82baf8f71b53 LogTimestamp=2021-05-28T01:03:00.577108Z ResponseDesc=Created Exiting. 2021-05-28T01:03:00.577Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:03:00.673Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:00.673572Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:03:00.673Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:00.673572Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:00.674Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-05-28T01:03:00.770Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1875dda3-15eb-452f-8af4-3c5f4b8b2b50 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:00.770757Z InvocationID=5fbd5165-d0be-47ee-82a0-161a7d525450 LogTimestamp=2021-05-28T01:03:00.770829Z Entering 2021-05-28T01:03:00.771Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1875dda3-15eb-452f-8af4-3c5f4b8b2b50 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:00.770757Z InvocationID=5fbd5165-d0be-47ee-82a0-161a7d525450 LogTimestamp=2021-05-28T01:03:00.770829Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:03:00.869Z [qtp580718781-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:00.771Z InvocationID=8b577777-90e2-4870-92a4-9717f1158c2c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id pomba 2021-05-28T01:03:02.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37d31172-0316-4db0-bf40-5ec726a69285 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:02.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37d31172-0316-4db0-bf40-5ec726a69285 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2340439a 2021-05-28T01:03:02.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37d31172-0316-4db0-bf40-5ec726a69285 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:02.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37d31172-0316-4db0-bf40-5ec726a69285 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:03.558Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2787 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:00.771Z InvocationID=8b577777-90e2-4870-92a4-9717f1158c2c LogTimestamp=2021-05-28T01:03:03.558491Z ResponseDesc=Created Exiting. 2021-05-28T01:03:03.558Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:03:03.572Z [qtp580718781-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:03.572246Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:03:03.572Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:03.572246Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:03.572Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-05-28T01:03:03.758Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f9576f0d-76d0-4d07-8dac-68780fbff223 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:03.758283Z InvocationID=b6410a78-6f01-4729-93a4-a02fde469ae2 LogTimestamp=2021-05-28T01:03:03.758337Z Entering 2021-05-28T01:03:03.758Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f9576f0d-76d0-4d07-8dac-68780fbff223 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:03.758283Z InvocationID=b6410a78-6f01-4729-93a4-a02fde469ae2 LogTimestamp=2021-05-28T01:03:03.758337Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:03:03.769Z [qtp580718781-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:03.758Z InvocationID=ba19fc96-cd49-4df3-8483-e660f849c873 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id multicloud 2021-05-28T01:03:04.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130f8696-bac5-4be7-91d1-20a5b678258f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:04.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130f8696-bac5-4be7-91d1-20a5b678258f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7246eca9 2021-05-28T01:03:04.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130f8696-bac5-4be7-91d1-20a5b678258f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:04.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130f8696-bac5-4be7-91d1-20a5b678258f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:03:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:03:07.774Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=4015 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:03.758Z InvocationID=ba19fc96-cd49-4df3-8483-e660f849c873 LogTimestamp=2021-05-28T01:03:07.773992Z ResponseDesc=Created Exiting. 2021-05-28T01:03:07.774Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:03:07.871Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:07.871145Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:03:07.871Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:07.871145Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:07.871Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-05-28T01:03:07.880Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d5107dca-a41c-4cb2-ab1e-266b7a80e284 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:07.880240Z InvocationID=6e9c52e3-99ac-4145-b38c-21b2c8fcf108 LogTimestamp=2021-05-28T01:03:07.880295Z Entering 2021-05-28T01:03:07.880Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d5107dca-a41c-4cb2-ab1e-266b7a80e284 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:07.880240Z InvocationID=6e9c52e3-99ac-4145-b38c-21b2c8fcf108 LogTimestamp=2021-05-28T01:03:07.880295Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:03:07.962Z [qtp580718781-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:07.880Z InvocationID=c666de38-7a44-4c68-9a51-8c26c090d9f4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id cds 2021-05-28T01:03:08.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3106371-0ae3-4663-8257-4684e2bd1b84 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:08.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3106371-0ae3-4663-8257-4684e2bd1b84 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@513c3a47 2021-05-28T01:03:08.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3106371-0ae3-4663-8257-4684e2bd1b84 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:08.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3106371-0ae3-4663-8257-4684e2bd1b84 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:09.463Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=1582 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:07.880Z InvocationID=c666de38-7a44-4c68-9a51-8c26c090d9f4 LogTimestamp=2021-05-28T01:03:09.462962Z ResponseDesc=Created Exiting. 2021-05-28T01:03:09.463Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:03:09.474Z [qtp580718781-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:09.474742Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:03:09.474Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:09.474742Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:09.475Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-05-28T01:03:09.573Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=484b3809-8aae-4a00-9dcc-abc6ea258e49 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:09.573605Z InvocationID=800a825b-cf66-4856-8989-7d7feb3c54b9 LogTimestamp=2021-05-28T01:03:09.573654Z Entering 2021-05-28T01:03:09.573Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=484b3809-8aae-4a00-9dcc-abc6ea258e49 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/consumers ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:09.573605Z InvocationID=800a825b-cf66-4856-8989-7d7feb3c54b9 LogTimestamp=2021-05-28T01:03:09.573654Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-05-28T01:03:09.678Z [qtp580718781-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:09.574Z InvocationID=fc56b2e7-fb7c-4a90-8ab4-ba0ed8383960 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk lock resource with id modeling 2021-05-28T01:03:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fee5d8f-77b2-4d5a-8952-73f8e083c653 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fee5d8f-77b2-4d5a-8952-73f8e083c653 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36e7504d 2021-05-28T01:03:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fee5d8f-77b2-4d5a-8952-73f8e083c653 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3fee5d8f-77b2-4d5a-8952-73f8e083c653 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:12.166Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2592 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/consumers EntryTimestamp=2021-05-28T01:03:09.574Z InvocationID=fc56b2e7-fb7c-4a90-8ab4-ba0ed8383960 LogTimestamp=2021-05-28T01:03:12.166596Z ResponseDesc=Created Exiting. 2021-05-28T01:03:12.167Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-05-28T01:03:12.575Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8e9749d2-ba09-4d2d-bee9-781954f3bf6c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:12.574885Z InvocationID=e006610e-e00c-4c58-9179-ffbe28a6b67f LogTimestamp=2021-05-28T01:03:12.574940Z Entering 2021-05-28T01:03:12.575Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8e9749d2-ba09-4d2d-bee9-781954f3bf6c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:12.574885Z InvocationID=e006610e-e00c-4c58-9179-ffbe28a6b67f LogTimestamp=2021-05-28T01:03:12.574940Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-05-28T01:03:12.674Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=99 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-05-28T01:03:12.575Z InvocationID=755bf29f-fb68-4219-8d86-9b87d54416b7 LogTimestamp=2021-05-28T01:03:12.674106Z ResponseDesc=OK Exiting. 2021-05-28T01:03:12.674Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-05-28T01:03:13.271Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a6cc3782-7f58-4cde-b5a2-fc8857fac61c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:13.271401Z InvocationID=c0fb3850-82e3-44eb-a676-32a14705bd81 LogTimestamp=2021-05-28T01:03:13.271459Z Entering 2021-05-28T01:03:13.271Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a6cc3782-7f58-4cde-b5a2-fc8857fac61c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/screen ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:13.271401Z InvocationID=c0fb3850-82e3-44eb-a676-32a14705bd81 LogTimestamp=2021-05-28T01:03:13.271459Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 GET /sdc2/rest/v1/screen HTTP/1.1 2021-05-28T01:03:14.070Z [qtp580718781-16] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T01:03:13.271Z InvocationID=3380e25e-6b97-457d-a150-e5c1d53051af PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk time to fetch all catalog elements: StopWatch '': running time = 796440806 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n796440806 100% \n 2021-05-28T01:03:14.081Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=810 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=COMPLETE ServerIPAddress=10.242.102.221 ServiceName=/v1/screen EntryTimestamp=2021-05-28T01:03:13.271Z InvocationID=3380e25e-6b97-457d-a150-e5c1d53051af LogTimestamp=2021-05-28T01:03:14.081163Z ResponseDesc=OK Exiting. 2021-05-28T01:03:14.081Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-05-28T01:03:14.459Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0b64d5fa-bfa4-459f-97a9-745ea4fa0c00 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:14.459399Z InvocationID=87b035cf-db9f-4928-b93f-2a3b551cdc43 LogTimestamp=2021-05-28T01:03:14.459434Z Entering 2021-05-28T01:03:14.459Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0b64d5fa-bfa4-459f-97a9-745ea4fa0c00 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:14.459399Z InvocationID=87b035cf-db9f-4928-b93f-2a3b551cdc43 LogTimestamp=2021-05-28T01:03:14.459434Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:03:14.561Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:03:14.459Z InvocationID=307fe189-b50e-4479-861f-6159d600302a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:03:14.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=973db6b2-98b6-4af5-a798-a658920e07ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:14.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=973db6b2-98b6-4af5-a798-a658920e07ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@171a9998 2021-05-28T01:03:14.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=973db6b2-98b6-4af5-a798-a658920e07ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:14.573Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=973db6b2-98b6-4af5-a798-a658920e07ec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:14.672Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:14.459Z InvocationID=307fe189-b50e-4479-861f-6159d600302a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:03:14.673Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:14.459Z InvocationID=307fe189-b50e-4479-861f-6159d600302a ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:03:14.673Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=214 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:03:14.459Z InvocationID=307fe189-b50e-4479-861f-6159d600302a ErrorCode=500 LogTimestamp=2021-05-28T01:03:14.673532Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:14.674Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:03:17.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38149414-103a-4ee8-946e-b9c3db66371d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:17.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38149414-103a-4ee8-946e-b9c3db66371d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@527c0c76 2021-05-28T01:03:17.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38149414-103a-4ee8-946e-b9c3db66371d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:17.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38149414-103a-4ee8-946e-b9c3db66371d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415233f2-b747-4999-904e-760dffc437d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415233f2-b747-4999-904e-760dffc437d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6222fa7c 2021-05-28T01:03:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415233f2-b747-4999-904e-760dffc437d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=415233f2-b747-4999-904e-760dffc437d4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:03:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:03:22.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29cafea-c632-430d-86ab-797c80c7b124 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:22.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29cafea-c632-430d-86ab-797c80c7b124 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e4bab4d 2021-05-28T01:03:22.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29cafea-c632-430d-86ab-797c80c7b124 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:22.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b29cafea-c632-430d-86ab-797c80c7b124 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:24.862Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=197591bc-62fa-4e5e-9ee5-323fc27d140f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:24.861872Z InvocationID=10221f20-e7f7-4a55-ad60-e767c7220966 LogTimestamp=2021-05-28T01:03:24.861929Z Entering 2021-05-28T01:03:24.862Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=197591bc-62fa-4e5e-9ee5-323fc27d140f InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:24.861872Z InvocationID=10221f20-e7f7-4a55-ad60-e767c7220966 LogTimestamp=2021-05-28T01:03:24.861929Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:03:24.974Z [qtp580718781-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:03:24.862Z InvocationID=b00db611-3f78-462b-81a8-62224dc9f947 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:03:25.055Z [qtp580718781-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:24.862Z InvocationID=b00db611-3f78-462b-81a8-62224dc9f947 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:03:25.056Z [qtp580718781-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:24.862Z InvocationID=b00db611-3f78-462b-81a8-62224dc9f947 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:03:25.057Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=194 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:03:24.862Z InvocationID=b00db611-3f78-462b-81a8-62224dc9f947 ErrorCode=500 LogTimestamp=2021-05-28T01:03:25.056997Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:25.057Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:03:26.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10c51824-9e88-48f4-944f-7a6036a9bdaa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:26.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10c51824-9e88-48f4-944f-7a6036a9bdaa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2dde2945 2021-05-28T01:03:26.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10c51824-9e88-48f4-944f-7a6036a9bdaa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:26.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10c51824-9e88-48f4-944f-7a6036a9bdaa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cbc2f0e-993b-49aa-9234-7642049306fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cbc2f0e-993b-49aa-9234-7642049306fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ff38f34 2021-05-28T01:03:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cbc2f0e-993b-49aa-9234-7642049306fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cbc2f0e-993b-49aa-9234-7642049306fd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:32.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954282e5-fd43-4cd7-b0af-cb6f64247b82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:32.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954282e5-fd43-4cd7-b0af-cb6f64247b82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a7767ac 2021-05-28T01:03:32.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954282e5-fd43-4cd7-b0af-cb6f64247b82 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:32.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954282e5-fd43-4cd7-b0af-cb6f64247b82 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:35.273Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ed22e38a-2dd3-4fe4-96f1-176108f99b14 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:35.273619Z InvocationID=0101d764-6fd7-4311-a9d3-3f7261890a31 LogTimestamp=2021-05-28T01:03:35.273707Z Entering 2021-05-28T01:03:35.273Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ed22e38a-2dd3-4fe4-96f1-176108f99b14 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:35.273619Z InvocationID=0101d764-6fd7-4311-a9d3-3f7261890a31 LogTimestamp=2021-05-28T01:03:35.273707Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:03:35.360Z [qtp580718781-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:03:35.274Z InvocationID=3536b46e-a3d8-418c-b13d-eafe0f25f70b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:03:35.468Z [qtp580718781-16] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:35.274Z InvocationID=3536b46e-a3d8-418c-b13d-eafe0f25f70b ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:03:35.468Z [qtp580718781-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:35.274Z InvocationID=3536b46e-a3d8-418c-b13d-eafe0f25f70b ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:03:35.469Z [qtp580718781-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=195 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:03:35.274Z InvocationID=3536b46e-a3d8-418c-b13d-eafe0f25f70b ErrorCode=500 LogTimestamp=2021-05-28T01:03:35.469284Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:35.469Z [qtp580718781-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:03:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:03:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:03:36.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1cca261-2e08-4eb5-8ed6-6bdc79b62ab7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:36.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1cca261-2e08-4eb5-8ed6-6bdc79b62ab7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5955b86c 2021-05-28T01:03:36.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1cca261-2e08-4eb5-8ed6-6bdc79b62ab7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:36.165Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1cca261-2e08-4eb5-8ed6-6bdc79b62ab7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:03:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:03:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:03:37.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cfe33d-4a11-4a64-abee-d789e52300ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:37.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cfe33d-4a11-4a64-abee-d789e52300ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51d2602 2021-05-28T01:03:37.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cfe33d-4a11-4a64-abee-d789e52300ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:37.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cfe33d-4a11-4a64-abee-d789e52300ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:41.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a2f6e7-9b29-447c-9d6b-546abc4098a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:03:41.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a2f6e7-9b29-447c-9d6b-546abc4098a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61493ba 2021-05-28T01:03:41.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a2f6e7-9b29-447c-9d6b-546abc4098a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:03:41.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a2f6e7-9b29-447c-9d6b-546abc4098a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:03:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:03:44.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:03:45.868Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2a9d16e9-5b64-463c-a438-c08b642d01e3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:45.867923Z InvocationID=77031150-231b-458e-93a2-fe0c53496dd5 LogTimestamp=2021-05-28T01:03:45.867971Z Entering 2021-05-28T01:03:45.868Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2a9d16e9-5b64-463c-a438-c08b642d01e3 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:45.867923Z InvocationID=77031150-231b-458e-93a2-fe0c53496dd5 LogTimestamp=2021-05-28T01:03:45.867971Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:03:45.869Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:45.868Z InvocationID=704cb844-9af7-470f-ac12-f5d976bf4391 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T01:03:45.870Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T01:03:45.869Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.183 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T01:03:45.868Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=704cb844-9af7-470f-ac12-f5d976bf4391 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@7562d55a 2021-05-28T01:03:45.870Z [qtp580718781-96] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T01:03:45.869Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.183 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T01:03:45.868Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=704cb844-9af7-470f-ac12-f5d976bf4391 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@7562d55a 2021-05-28T01:03:45.871Z [qtp580718781-96] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:45.870814Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:03:45.871Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:45.870814Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:45.871Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:03:47.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:03:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:03:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:03:50.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:03:51.070Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: java.lang.RuntimeException: org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:03:51.571Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: java.lang.RuntimeException: org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:03:52.114Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:03:53.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:03:55.961Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=534d8bb8-dfc9-4f95-a11d-3cf89ebb785d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:55.961392Z InvocationID=affe1450-111c-40f6-9a5b-90e98c339a2f LogTimestamp=2021-05-28T01:03:55.961466Z Entering 2021-05-28T01:03:55.961Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=534d8bb8-dfc9-4f95-a11d-3cf89ebb785d InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:03:55.961392Z InvocationID=affe1450-111c-40f6-9a5b-90e98c339a2f LogTimestamp=2021-05-28T01:03:55.961466Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:03:55.963Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:03:55.962Z InvocationID=b626eb64-83f6-4880-b01d-2f9bd3d371d7 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-05-28T01:03:55.964Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T01:03:55.963Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.183 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T01:03:55.962Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=b626eb64-83f6-4880-b01d-2f9bd3d371d7 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5827bc7f 2021-05-28T01:03:55.964Z [qtp580718781-15] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=1 EndTimestamp=2021-05-28T01:03:55.963Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=null RemoteHost=10.242.170.183 ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-05-28T01:03:55.962Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=b626eb64-83f6-4880-b01d-2f9bd3d371d7 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5827bc7f 2021-05-28T01:03:55.964Z [qtp580718781-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:55.964678Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-05-28T01:03:55.965Z [qtp580718781-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.102.221 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-05-28T01:03:55.964678Z ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:03:55.965Z [qtp580718781-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:03:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:03:57.672Z [cluster1-reconnection-0] ERROR c.d.d.c.AbstractReconnectionHandler Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. 2021-05-28T01:03:57.673Z [cluster1-reconnection-0] ERROR com.datastax.driver.core.Cluster Authentication error during reconnection to cassandra.onap/10.242.170.166:9042, scheduling retry in 2000 milliseconds com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:03:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:04:01.667Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:04:01.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a3377a-f5be-489b-a46c-c831ec17c5e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:01.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a3377a-f5be-489b-a46c-c831ec17c5e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@578fded3 2021-05-28T01:04:01.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a3377a-f5be-489b-a46c-c831ec17c5e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:01.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a3377a-f5be-489b-a46c-c831ec17c5e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:03.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:03.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23ed8a96 2021-05-28T01:04:03.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:03.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:04:05.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:04:06.666Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:04:06.666Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:04:06.666Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:04:06.667Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a069a447-7c65-4cf6-a46b-b28c59e07302 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:04:09.178Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a52cb180-a1bd-4748-aa38-622a75a76656 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:09.177608Z InvocationID=5485a6e6-0def-431e-aa89-be69d6eb892e LogTimestamp=2021-05-28T01:04:09.177706Z Entering 2021-05-28T01:04:09.178Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a52cb180-a1bd-4748-aa38-622a75a76656 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:09.177608Z InvocationID=5485a6e6-0def-431e-aa89-be69d6eb892e LogTimestamp=2021-05-28T01:04:09.177706Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:04:09.277Z [qtp580718781-19] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:04:09.178Z InvocationID=ea888b40-17db-4914-b5ad-7671ea563381 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:04:09.461Z [qtp580718781-19] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:09.178Z InvocationID=ea888b40-17db-4914-b5ad-7671ea563381 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:04:09.462Z [qtp580718781-19] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:09.178Z InvocationID=ea888b40-17db-4914-b5ad-7671ea563381 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:04:09.463Z [qtp580718781-19] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=285 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:04:09.178Z InvocationID=ea888b40-17db-4914-b5ad-7671ea563381 ErrorCode=500 LogTimestamp=2021-05-28T01:04:09.463247Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:04:09.463Z [qtp580718781-19] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:04:10.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9657c5dc-f841-4920-a17e-c898fb61a9fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:10.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9657c5dc-f841-4920-a17e-c898fb61a9fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fac95df 2021-05-28T01:04:10.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9657c5dc-f841-4920-a17e-c898fb61a9fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:10.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9657c5dc-f841-4920-a17e-c898fb61a9fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:12.175Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9657c5dc-f841-4920-a17e-c898fb61a9fe ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:04:12.175Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9657c5dc-f841-4920-a17e-c898fb61a9fe ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:04:12.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4bacd-7170-4e95-a8fb-3ec2ada76aba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:12.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4bacd-7170-4e95-a8fb-3ec2ada76aba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@445774f6 2021-05-28T01:04:12.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4bacd-7170-4e95-a8fb-3ec2ada76aba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:12.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f4bacd-7170-4e95-a8fb-3ec2ada76aba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:14.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad90c46-6578-4916-ad43-c2e1603eb0f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:14.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad90c46-6578-4916-ad43-c2e1603eb0f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ba1ffd0 2021-05-28T01:04:14.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad90c46-6578-4916-ad43-c2e1603eb0f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:14.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad90c46-6578-4916-ad43-c2e1603eb0f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:15.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b6d2f1f-3f19-4ad2-b6e9-aad1894c4e47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:15.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b6d2f1f-3f19-4ad2-b6e9-aad1894c4e47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20889116 2021-05-28T01:04:15.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b6d2f1f-3f19-4ad2-b6e9-aad1894c4e47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:15.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b6d2f1f-3f19-4ad2-b6e9-aad1894c4e47 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:16.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82bea05a-5323-408f-acca-66a823bc8590 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:16.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82bea05a-5323-408f-acca-66a823bc8590 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@776e6a1a 2021-05-28T01:04:16.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82bea05a-5323-408f-acca-66a823bc8590 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:16.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82bea05a-5323-408f-acca-66a823bc8590 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:18.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=454c1128-b838-4b37-83c7-63955bb1294b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:18.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=454c1128-b838-4b37-83c7-63955bb1294b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@86140c5 2021-05-28T01:04:18.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=454c1128-b838-4b37-83c7-63955bb1294b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:18.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=454c1128-b838-4b37-83c7-63955bb1294b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:19.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29793c96-643b-4833-bb35-f210fdb9e2bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:19.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29793c96-643b-4833-bb35-f210fdb9e2bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c3a6166 2021-05-28T01:04:19.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29793c96-643b-4833-bb35-f210fdb9e2bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:19.182Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29793c96-643b-4833-bb35-f210fdb9e2bb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:19.584Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e589a618-8a4b-4ad8-b573-eac989fb9bb4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:19.584326Z InvocationID=37cb27ff-2839-4dd7-8dfc-178cfb7e44c5 LogTimestamp=2021-05-28T01:04:19.584414Z Entering 2021-05-28T01:04:19.584Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e589a618-8a4b-4ad8-b573-eac989fb9bb4 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:19.584326Z InvocationID=37cb27ff-2839-4dd7-8dfc-178cfb7e44c5 LogTimestamp=2021-05-28T01:04:19.584414Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:04:19.689Z [qtp580718781-12] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:04:19.585Z InvocationID=f9844a4d-d04b-406c-a51d-90d863ce098a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:04:19.768Z [qtp580718781-12] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:19.585Z InvocationID=f9844a4d-d04b-406c-a51d-90d863ce098a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:04:19.768Z [qtp580718781-12] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:19.585Z InvocationID=f9844a4d-d04b-406c-a51d-90d863ce098a ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:04:19.769Z [qtp580718781-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=184 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:04:19.585Z InvocationID=f9844a4d-d04b-406c-a51d-90d863ce098a ErrorCode=500 LogTimestamp=2021-05-28T01:04:19.769205Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:04:19.769Z [qtp580718781-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:04:20.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d41274-6885-4b76-bed3-78b36b9ebaa1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:20.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d41274-6885-4b76-bed3-78b36b9ebaa1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@685e41d6 2021-05-28T01:04:20.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d41274-6885-4b76-bed3-78b36b9ebaa1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:20.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d41274-6885-4b76-bed3-78b36b9ebaa1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:04:20.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:04:21.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cab4b0a7-3112-4270-a8b1-c6855a7c7c12 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:21.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cab4b0a7-3112-4270-a8b1-c6855a7c7c12 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ce4c87d 2021-05-28T01:04:21.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cab4b0a7-3112-4270-a8b1-c6855a7c7c12 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:21.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cab4b0a7-3112-4270-a8b1-c6855a7c7c12 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:22.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf52732d-17e4-4f8f-832d-93283f8adaee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:22.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf52732d-17e4-4f8f-832d-93283f8adaee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e81da4c 2021-05-28T01:04:22.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf52732d-17e4-4f8f-832d-93283f8adaee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:22.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf52732d-17e4-4f8f-832d-93283f8adaee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:23.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b340fb2b-f27c-495b-88e0-5f3cedeb847b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:23.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b340fb2b-f27c-495b-88e0-5f3cedeb847b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@316fa7ab 2021-05-28T01:04:23.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b340fb2b-f27c-495b-88e0-5f3cedeb847b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:23.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b340fb2b-f27c-495b-88e0-5f3cedeb847b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:25.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30c994d7-d388-4952-8895-5eabfd614b66 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:25.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30c994d7-d388-4952-8895-5eabfd614b66 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ba500a1 2021-05-28T01:04:25.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30c994d7-d388-4952-8895-5eabfd614b66 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:25.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30c994d7-d388-4952-8895-5eabfd614b66 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:26.375Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0778d1b3-8707-478e-8715-5ed91c283958 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:26.375Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0778d1b3-8707-478e-8715-5ed91c283958 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61c89288 2021-05-28T01:04:26.375Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0778d1b3-8707-478e-8715-5ed91c283958 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:26.376Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0778d1b3-8707-478e-8715-5ed91c283958 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:28.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca73e3d-b8c1-4a0c-bdec-344474652973 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:28.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca73e3d-b8c1-4a0c-bdec-344474652973 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17b7747 2021-05-28T01:04:28.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca73e3d-b8c1-4a0c-bdec-344474652973 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:28.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca73e3d-b8c1-4a0c-bdec-344474652973 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:29.865Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=eda89538-8824-4b2b-b00d-207ea489445a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:29.865297Z InvocationID=e3a30c72-23e3-40c0-8a9a-97419bda9948 LogTimestamp=2021-05-28T01:04:29.865356Z Entering 2021-05-28T01:04:29.865Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=eda89538-8824-4b2b-b00d-207ea489445a InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:29.865297Z InvocationID=e3a30c72-23e3-40c0-8a9a-97419bda9948 LogTimestamp=2021-05-28T01:04:29.865356Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:04:29.956Z [qtp580718781-96] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:04:29.865Z InvocationID=ac564247-3b68-4d77-b4a8-68d6df4c771a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:04:29.971Z [qtp580718781-96] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:29.865Z InvocationID=ac564247-3b68-4d77-b4a8-68d6df4c771a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:04:29.971Z [qtp580718781-96] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:29.865Z InvocationID=ac564247-3b68-4d77-b4a8-68d6df4c771a ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:04:29.972Z [qtp580718781-96] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=107 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:04:29.865Z InvocationID=ac564247-3b68-4d77-b4a8-68d6df4c771a ErrorCode=500 LogTimestamp=2021-05-28T01:04:29.972074Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:04:29.972Z [qtp580718781-96] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:04:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=386c758f-0bb1-4e7b-a3ec-1e3bb109d015 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=386c758f-0bb1-4e7b-a3ec-1e3bb109d015 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d65ca3c 2021-05-28T01:04:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=386c758f-0bb1-4e7b-a3ec-1e3bb109d015 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=386c758f-0bb1-4e7b-a3ec-1e3bb109d015 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d947e8da-7839-4bd5-b86a-a79e66153317 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d947e8da-7839-4bd5-b86a-a79e66153317 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e467b39 2021-05-28T01:04:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d947e8da-7839-4bd5-b86a-a79e66153317 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d947e8da-7839-4bd5-b86a-a79e66153317 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:04:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:04:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:04:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:04:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:04:37.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2b2ae-1403-4924-a0d1-e3d83b0272fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:37.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2b2ae-1403-4924-a0d1-e3d83b0272fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b3ffcab 2021-05-28T01:04:37.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2b2ae-1403-4924-a0d1-e3d83b0272fc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:37.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2b2ae-1403-4924-a0d1-e3d83b0272fc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:40.077Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a92e64c4-37b4-46bb-9505-2827ec5fc34c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:40.077022Z InvocationID=131a6dfe-394d-4a42-94ee-13629016a03f LogTimestamp=2021-05-28T01:04:40.077073Z Entering 2021-05-28T01:04:40.077Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a92e64c4-37b4-46bb-9505-2827ec5fc34c InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:40.077022Z InvocationID=131a6dfe-394d-4a42-94ee-13629016a03f LogTimestamp=2021-05-28T01:04:40.077073Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:04:40.173Z [qtp580718781-98] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:04:40.077Z InvocationID=6242de07-d8bc-402a-9f06-a6897231da68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:04:40.259Z [qtp580718781-98] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:40.077Z InvocationID=6242de07-d8bc-402a-9f06-a6897231da68 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:04:40.259Z [qtp580718781-98] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:40.077Z InvocationID=6242de07-d8bc-402a-9f06-a6897231da68 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:04:40.260Z [qtp580718781-98] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=183 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:04:40.077Z InvocationID=6242de07-d8bc-402a-9f06-a6897231da68 ErrorCode=500 LogTimestamp=2021-05-28T01:04:40.260171Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:04:40.260Z [qtp580718781-98] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:04:40.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e297edf-8d41-457d-a9d9-ea3859c01788 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:40.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e297edf-8d41-457d-a9d9-ea3859c01788 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31a47fc4 2021-05-28T01:04:40.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e297edf-8d41-457d-a9d9-ea3859c01788 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:40.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e297edf-8d41-457d-a9d9-ea3859c01788 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:43.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1efd5f7-a09b-49b0-8d1f-f79166649101 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:43.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1efd5f7-a09b-49b0-8d1f-f79166649101 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@527367e6 2021-05-28T01:04:43.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1efd5f7-a09b-49b0-8d1f-f79166649101 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:43.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1efd5f7-a09b-49b0-8d1f-f79166649101 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:46.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7298c69a-dc79-4d21-b75d-1db36f7e287d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:46.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7298c69a-dc79-4d21-b75d-1db36f7e287d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f915122 2021-05-28T01:04:46.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7298c69a-dc79-4d21-b75d-1db36f7e287d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:46.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7298c69a-dc79-4d21-b75d-1db36f7e287d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:49.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97d6f3f4-a63e-45e6-9165-3940b86d35d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:49.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97d6f3f4-a63e-45e6-9165-3940b86d35d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bdf3adf 2021-05-28T01:04:49.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97d6f3f4-a63e-45e6-9165-3940b86d35d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:49.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97d6f3f4-a63e-45e6-9165-3940b86d35d5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:50.281Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c0e9ba83-25c6-4ac8-be30-bc0e9381dadb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:50.281468Z InvocationID=a65ab1c5-1669-4cfe-851f-a98144d33d78 LogTimestamp=2021-05-28T01:04:50.281523Z Entering 2021-05-28T01:04:50.281Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c0e9ba83-25c6-4ac8-be30-bc0e9381dadb InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.102.221 ServiceInstanceID=null EntryTimestamp=2021-05-28T01:04:50.281468Z InvocationID=a65ab1c5-1669-4cfe-851f-a98144d33d78 LogTimestamp=2021-05-28T01:04:50.281523Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:04:50.375Z [qtp580718781-11] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:04:50.281Z InvocationID=bfbfdcb5-76df-4a2c-958c-161917d2e763 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:04:50.460Z [qtp580718781-11] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:50.281Z InvocationID=bfbfdcb5-76df-4a2c-958c-161917d2e763 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:04:50.460Z [qtp580718781-11] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:04:50.281Z InvocationID=bfbfdcb5-76df-4a2c-958c-161917d2e763 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:04:50.460Z [qtp580718781-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=179 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:04:50.281Z InvocationID=bfbfdcb5-76df-4a2c-958c-161917d2e763 ErrorCode=500 LogTimestamp=2021-05-28T01:04:50.460807Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:04:50.461Z [qtp580718781-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:04:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:04:50.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:04:52.120Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:04:52.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e42b2c-fc82-4bad-86ea-e97a8d55aa83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:52.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e42b2c-fc82-4bad-86ea-e97a8d55aa83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2997383f 2021-05-28T01:04:52.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e42b2c-fc82-4bad-86ea-e97a8d55aa83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:52.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94e42b2c-fc82-4bad-86ea-e97a8d55aa83 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53ccd981-354b-4585-b3ce-d28d5cf4fe2f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53ccd981-354b-4585-b3ce-d28d5cf4fe2f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@609226bc 2021-05-28T01:04:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53ccd981-354b-4585-b3ce-d28d5cf4fe2f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53ccd981-354b-4585-b3ce-d28d5cf4fe2f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:04:58.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ef50552-3b5e-4dae-ae2a-e9b9fdfc966a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:04:58.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ef50552-3b5e-4dae-ae2a-e9b9fdfc966a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45af73cd 2021-05-28T01:04:58.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ef50552-3b5e-4dae-ae2a-e9b9fdfc966a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:04:58.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ef50552-3b5e-4dae-ae2a-e9b9fdfc966a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:00.561Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d634339a-478d-4d33-aefe-15adac60ed43 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T01:05:00.560688Z InvocationID=7e272088-0fa0-4708-9fb0-ec4f409ff89b LogTimestamp=2021-05-28T01:05:00.560788Z Entering 2021-05-28T01:05:00.561Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d634339a-478d-4d33-aefe-15adac60ed43 InstanceID=57da187e-ef32-4c59-a3f4-fedf679550b6 ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk ClientIPAddress=10.242.170.183 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.170.183 ServerIPAddress=10.242.102.221 ServiceName=v1/catalog/uploadType/datatypes ServiceInstanceID=null EntryTimestamp=2021-05-28T01:05:00.560688Z InvocationID=7e272088-0fa0-4708-9fb0-ec4f409ff89b LogTimestamp=2021-05-28T01:05:00.560788Z localAddr=10.242.102.221 remoteAddr=10.242.170.183 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-05-28T01:05:00.658Z [qtp580718781-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-05-28T01:05:00.561Z InvocationID=cd8037f8-dad1-4ac3-88c7-f9b256069d91 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-05-28T01:05:00.677Z [qtp580718781-17] ERROR o.o.s.b.d.j.JanusGraphGenericDao RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:05:00.561Z InvocationID=cd8037f8-dad1-4ac3-88c7-f9b256069d91 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=null Going to execute rollback on graph. 2021-05-28T01:05:00.677Z [qtp580718781-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-05-28T01:05:00.561Z InvocationID=cd8037f8-dad1-4ac3-88c7-f9b256069d91 ErrorCode=900 Unexpected error during BE REST API execution 2021-05-28T01:05:00.678Z [qtp580718781-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=117 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-7697c4f957-9fbvk StatusCode=ERROR TargetEntity=Create dataType ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-05-28T01:05:00.561Z InvocationID=cd8037f8-dad1-4ac3-88c7-f9b256069d91 ErrorCode=500 LogTimestamp=2021-05-28T01:05:00.678088Z ResponseDesc=Internal Server Error Exiting. 2021-05-28T01:05:00.678Z [qtp580718781-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="500" 2021-05-28T01:05:01.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1562b4e-1452-40d3-9b42-8e2ed98911fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:01.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1562b4e-1452-40d3-9b42-8e2ed98911fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@764b065 2021-05-28T01:05:01.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1562b4e-1452-40d3-9b42-8e2ed98911fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:01.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1562b4e-1452-40d3-9b42-8e2ed98911fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:04.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80fad1fd-3416-41a3-9b3b-6ba1e003086f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:04.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80fad1fd-3416-41a3-9b3b-6ba1e003086f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17a23676 2021-05-28T01:05:04.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80fad1fd-3416-41a3-9b3b-6ba1e003086f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:04.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80fad1fd-3416-41a3-9b3b-6ba1e003086f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:05:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:05:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d03e2290-1238-47b2-adf5-bcf5d322020f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d03e2290-1238-47b2-adf5-bcf5d322020f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66f455f8 2021-05-28T01:05:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d03e2290-1238-47b2-adf5-bcf5d322020f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:07.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d03e2290-1238-47b2-adf5-bcf5d322020f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:10.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c88ad79-056f-41cb-89a0-1263be0cf2a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:10.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c88ad79-056f-41cb-89a0-1263be0cf2a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58fb63f0 2021-05-28T01:05:10.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c88ad79-056f-41cb-89a0-1263be0cf2a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:10.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c88ad79-056f-41cb-89a0-1263be0cf2a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:13.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fef367b5-87ce-4fe6-a59e-16e563207818 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:13.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fef367b5-87ce-4fe6-a59e-16e563207818 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7abe0712 2021-05-28T01:05:13.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fef367b5-87ce-4fe6-a59e-16e563207818 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:13.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fef367b5-87ce-4fe6-a59e-16e563207818 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bbec39e-0c67-474a-b4f0-257888116765 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bbec39e-0c67-474a-b4f0-257888116765 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ca99fb5 2021-05-28T01:05:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bbec39e-0c67-474a-b4f0-257888116765 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bbec39e-0c67-474a-b4f0-257888116765 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab2877b2-d4f6-44dd-a45e-88b49d649e1a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab2877b2-d4f6-44dd-a45e-88b49d649e1a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38b65e4d 2021-05-28T01:05:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab2877b2-d4f6-44dd-a45e-88b49d649e1a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:19.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab2877b2-d4f6-44dd-a45e-88b49d649e1a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:05:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:05:22.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33708cee-8911-4201-8a8a-767833690373 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:22.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33708cee-8911-4201-8a8a-767833690373 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b515e48 2021-05-28T01:05:22.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33708cee-8911-4201-8a8a-767833690373 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:22.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33708cee-8911-4201-8a8a-767833690373 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ffb04a-e897-4b91-84c8-f71e0d8630c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ffb04a-e897-4b91-84c8-f71e0d8630c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@315d48a7 2021-05-28T01:05:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ffb04a-e897-4b91-84c8-f71e0d8630c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09ffb04a-e897-4b91-84c8-f71e0d8630c4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:28.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1675e592-6428-48dc-9f2e-b4934fe0e6d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:28.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1675e592-6428-48dc-9f2e-b4934fe0e6d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1393d0e 2021-05-28T01:05:28.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1675e592-6428-48dc-9f2e-b4934fe0e6d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:28.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1675e592-6428-48dc-9f2e-b4934fe0e6d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:31.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f8ad78b-c80a-4c0a-b013-f34a31fa97bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:31.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f8ad78b-c80a-4c0a-b013-f34a31fa97bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bf75f52 2021-05-28T01:05:31.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f8ad78b-c80a-4c0a-b013-f34a31fa97bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:31.788Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f8ad78b-c80a-4c0a-b013-f34a31fa97bf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:35.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a967428d-48c5-44f5-95fd-3d40e74836fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:35.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a967428d-48c5-44f5-95fd-3d40e74836fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ae5e37a 2021-05-28T01:05:35.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a967428d-48c5-44f5-95fd-3d40e74836fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:35.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a967428d-48c5-44f5-95fd-3d40e74836fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:05:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:05:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:05:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:05:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:05:38.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0115441c-be60-4679-8422-cdb72e64b5ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:38.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0115441c-be60-4679-8422-cdb72e64b5ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61a69cf1 2021-05-28T01:05:38.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0115441c-be60-4679-8422-cdb72e64b5ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:38.085Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0115441c-be60-4679-8422-cdb72e64b5ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@293f1c86 2021-05-28T01:05:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:43.872Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:05:43.873Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:05:43.874Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:05:43.874Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=5bff0afd-4a0d-42dc-922b-1d6ef669351d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:05:46.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca6bad4a-301f-4c4c-81ac-418e3710827c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:46.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca6bad4a-301f-4c4c-81ac-418e3710827c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1883cb3c 2021-05-28T01:05:46.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca6bad4a-301f-4c4c-81ac-418e3710827c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:46.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca6bad4a-301f-4c4c-81ac-418e3710827c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:46.572Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ca6bad4a-301f-4c4c-81ac-418e3710827c ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:05:46.572Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ca6bad4a-301f-4c4c-81ac-418e3710827c ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:05:47.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=255466d4-70ad-4190-991b-0f940baebd83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:47.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=255466d4-70ad-4190-991b-0f940baebd83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@587c413d 2021-05-28T01:05:47.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=255466d4-70ad-4190-991b-0f940baebd83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:47.078Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=255466d4-70ad-4190-991b-0f940baebd83 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:50.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79fda2f3-3fd1-4dc8-94ff-1fe2c0ffd6a7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:50.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79fda2f3-3fd1-4dc8-94ff-1fe2c0ffd6a7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9f6bd98 2021-05-28T01:05:50.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79fda2f3-3fd1-4dc8-94ff-1fe2c0ffd6a7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:50.257Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79fda2f3-3fd1-4dc8-94ff-1fe2c0ffd6a7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:05:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:05:52.136Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:05:52.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a42a3668-b48a-453d-be45-3a9daf518704 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:52.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a42a3668-b48a-453d-be45-3a9daf518704 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f2a4db1 2021-05-28T01:05:52.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a42a3668-b48a-453d-be45-3a9daf518704 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:52.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a42a3668-b48a-453d-be45-3a9daf518704 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=432a6a7c-03fa-4387-9fcb-8938bae05317 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=432a6a7c-03fa-4387-9fcb-8938bae05317 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4224a21c 2021-05-28T01:05:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=432a6a7c-03fa-4387-9fcb-8938bae05317 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=432a6a7c-03fa-4387-9fcb-8938bae05317 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:05:59.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cc0d1c6-369f-4864-8b84-6ae9ffe33895 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:05:59.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cc0d1c6-369f-4864-8b84-6ae9ffe33895 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78eb66cc 2021-05-28T01:05:59.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cc0d1c6-369f-4864-8b84-6ae9ffe33895 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:05:59.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cc0d1c6-369f-4864-8b84-6ae9ffe33895 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:01.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2df13277-58f9-4851-ae23-706f69f79595 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2df13277-58f9-4851-ae23-706f69f79595 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@444d0e34 2021-05-28T01:06:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2df13277-58f9-4851-ae23-706f69f79595 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:01.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2df13277-58f9-4851-ae23-706f69f79595 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:06:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:06:05.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25ca6c41-3791-4bc2-87af-0f15e92d2f4e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:05.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25ca6c41-3791-4bc2-87af-0f15e92d2f4e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@195794c9 2021-05-28T01:06:05.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25ca6c41-3791-4bc2-87af-0f15e92d2f4e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:05.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25ca6c41-3791-4bc2-87af-0f15e92d2f4e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:08.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c56a133-f861-41ef-837a-e671425c74e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:08.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c56a133-f861-41ef-837a-e671425c74e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e84f1bd 2021-05-28T01:06:08.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c56a133-f861-41ef-837a-e671425c74e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:08.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c56a133-f861-41ef-837a-e671425c74e9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:11.059Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=8c56a133-f861-41ef-837a-e671425c74e9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:06:11.060Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=8c56a133-f861-41ef-837a-e671425c74e9 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:06:11.060Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=8c56a133-f861-41ef-837a-e671425c74e9 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:11.061Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=8c56a133-f861-41ef-837a-e671425c74e9 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:15.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b3980-3aa9-436c-ae09-de74ac1d88c8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:15.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b3980-3aa9-436c-ae09-de74ac1d88c8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d7a25a7 2021-05-28T01:06:15.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b3980-3aa9-436c-ae09-de74ac1d88c8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:15.457Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791b3980-3aa9-436c-ae09-de74ac1d88c8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:15.765Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=791b3980-3aa9-436c-ae09-de74ac1d88c8 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:15.765Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=791b3980-3aa9-436c-ae09-de74ac1d88c8 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57e6f9bd-8dcc-4d21-b4e2-605afca4e421 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57e6f9bd-8dcc-4d21-b4e2-605afca4e421 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@543e8fba 2021-05-28T01:06:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57e6f9bd-8dcc-4d21-b4e2-605afca4e421 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57e6f9bd-8dcc-4d21-b4e2-605afca4e421 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:17.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120c9938-c512-439f-9785-32491ae547e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:17.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120c9938-c512-439f-9785-32491ae547e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d6e8599 2021-05-28T01:06:17.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120c9938-c512-439f-9785-32491ae547e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:17.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120c9938-c512-439f-9785-32491ae547e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:20.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a5f9167-2d21-4840-8fe3-081882e3c12a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:20.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a5f9167-2d21-4840-8fe3-081882e3c12a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b5de8e 2021-05-28T01:06:20.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a5f9167-2d21-4840-8fe3-081882e3c12a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:20.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a5f9167-2d21-4840-8fe3-081882e3c12a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:06:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:06:22.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccdbd0ba-9414-47dc-96e8-60db1e9f5d66 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:22.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccdbd0ba-9414-47dc-96e8-60db1e9f5d66 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@251ffc06 2021-05-28T01:06:22.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccdbd0ba-9414-47dc-96e8-60db1e9f5d66 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:22.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccdbd0ba-9414-47dc-96e8-60db1e9f5d66 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:25.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d6eafa8-31d1-4514-a940-0f1ed4fe8581 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:25.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d6eafa8-31d1-4514-a940-0f1ed4fe8581 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57cf440 2021-05-28T01:06:25.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d6eafa8-31d1-4514-a940-0f1ed4fe8581 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:25.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1d6eafa8-31d1-4514-a940-0f1ed4fe8581 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:29.207Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3589725-65e1-4073-9d48-476d92f7218e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:29.207Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3589725-65e1-4073-9d48-476d92f7218e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7cc7eb3b 2021-05-28T01:06:29.207Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3589725-65e1-4073-9d48-476d92f7218e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:29.207Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3589725-65e1-4073-9d48-476d92f7218e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:32.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa51038-ea13-499b-b356-cc6955b4d1ac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:32.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa51038-ea13-499b-b356-cc6955b4d1ac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53290b83 2021-05-28T01:06:32.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa51038-ea13-499b-b356-cc6955b4d1ac PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:32.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa51038-ea13-499b-b356-cc6955b4d1ac PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bde8dc2-1f72-4b85-bbf3-ad7b8219daa1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bde8dc2-1f72-4b85-bbf3-ad7b8219daa1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@98c222c 2021-05-28T01:06:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bde8dc2-1f72-4b85-bbf3-ad7b8219daa1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:34.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bde8dc2-1f72-4b85-bbf3-ad7b8219daa1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:06:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:06:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:06:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:06:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:06:38.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:38.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b3019a9 2021-05-28T01:06:38.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:38.064Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:41.070Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:06:41.071Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:06:41.071Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:41.071Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=15e99c88-f7e8-4ce9-98c4-82fbedb17761 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:06:41.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:06:41.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0d8eb79-0e61-4495-8a25-f289d3947e47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:41.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0d8eb79-0e61-4495-8a25-f289d3947e47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28eda9c7 2021-05-28T01:06:41.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0d8eb79-0e61-4495-8a25-f289d3947e47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:41.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0d8eb79-0e61-4495-8a25-f289d3947e47 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:42.773Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c0d8eb79-0e61-4495-8a25-f289d3947e47 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:42.773Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c0d8eb79-0e61-4495-8a25-f289d3947e47 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:44.166Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:06:44.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e82c1dbb-7ea2-4f0e-89ca-abfa32a2d5c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:44.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e82c1dbb-7ea2-4f0e-89ca-abfa32a2d5c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b760344 2021-05-28T01:06:44.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e82c1dbb-7ea2-4f0e-89ca-abfa32a2d5c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:44.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e82c1dbb-7ea2-4f0e-89ca-abfa32a2d5c2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:45.259Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e82c1dbb-7ea2-4f0e-89ca-abfa32a2d5c2 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:45.259Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=e82c1dbb-7ea2-4f0e-89ca-abfa32a2d5c2 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:06:46.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b0f860b-7d73-42a5-b63f-f782ff1a96c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:46.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b0f860b-7d73-42a5-b63f-f782ff1a96c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30e0b4d2 2021-05-28T01:06:46.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b0f860b-7d73-42a5-b63f-f782ff1a96c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:46.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b0f860b-7d73-42a5-b63f-f782ff1a96c3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:48.173Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2b0f860b-7d73-42a5-b63f-f782ff1a96c3 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:48.174Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2b0f860b-7d73-42a5-b63f-f782ff1a96c3 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:06:50.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9cab0c8-f547-4f9e-b06b-a2aa093e2a12 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:50.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9cab0c8-f547-4f9e-b06b-a2aa093e2a12 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75869098 2021-05-28T01:06:50.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9cab0c8-f547-4f9e-b06b-a2aa093e2a12 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:50.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9cab0c8-f547-4f9e-b06b-a2aa093e2a12 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:06:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:06:52.117Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d79ee8-98f9-4f59-a35b-a21c5ca792d6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d79ee8-98f9-4f59-a35b-a21c5ca792d6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@587ac5e8 2021-05-28T01:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d79ee8-98f9-4f59-a35b-a21c5ca792d6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6d79ee8-98f9-4f59-a35b-a21c5ca792d6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:56.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d84c15de-07d2-4c27-bbf4-58233c096a20 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:56.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d84c15de-07d2-4c27-bbf4-58233c096a20 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@337d8180 2021-05-28T01:06:56.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d84c15de-07d2-4c27-bbf4-58233c096a20 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:56.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d84c15de-07d2-4c27-bbf4-58233c096a20 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:06:58.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acaf7c05-37f6-40c0-8c39-1216b4c96ad7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:06:58.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acaf7c05-37f6-40c0-8c39-1216b4c96ad7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68ee519 2021-05-28T01:06:58.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acaf7c05-37f6-40c0-8c39-1216b4c96ad7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:06:58.722Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acaf7c05-37f6-40c0-8c39-1216b4c96ad7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:02.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35ab4f-27ab-4880-b14b-0cb9ae069801 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:02.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35ab4f-27ab-4880-b14b-0cb9ae069801 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47cda700 2021-05-28T01:07:02.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35ab4f-27ab-4880-b14b-0cb9ae069801 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:02.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b35ab4f-27ab-4880-b14b-0cb9ae069801 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:07:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:07:06.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca3c476e-45d0-4a8e-a46b-51299f1bf442 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:06.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca3c476e-45d0-4a8e-a46b-51299f1bf442 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49166f94 2021-05-28T01:07:06.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca3c476e-45d0-4a8e-a46b-51299f1bf442 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:06.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca3c476e-45d0-4a8e-a46b-51299f1bf442 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:10.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:10.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16b896b7 2021-05-28T01:07:10.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:10.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:13.879Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:07:13.880Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:07:13.880Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:13.880Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3be4cb72-a7f7-416d-9371-54f250a11025 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:18.960Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:07:18.969Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:07:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:07:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:07:25.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7adac475-f54f-492a-96f9-5f8b2aec3aa4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:25.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7adac475-f54f-492a-96f9-5f8b2aec3aa4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67ef0f54 2021-05-28T01:07:25.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7adac475-f54f-492a-96f9-5f8b2aec3aa4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:25.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7adac475-f54f-492a-96f9-5f8b2aec3aa4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:26.689Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7adac475-f54f-492a-96f9-5f8b2aec3aa4 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:26.689Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7adac475-f54f-492a-96f9-5f8b2aec3aa4 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:28.025Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0a08c4a-b9fe-4b1c-8f35-547dc9a217a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:28.025Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0a08c4a-b9fe-4b1c-8f35-547dc9a217a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3720ede3 2021-05-28T01:07:28.025Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0a08c4a-b9fe-4b1c-8f35-547dc9a217a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:28.025Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0a08c4a-b9fe-4b1c-8f35-547dc9a217a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:31.209Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=968ac513-e15f-475a-97ba-3b784ff97811 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:31.209Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=968ac513-e15f-475a-97ba-3b784ff97811 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@657fc89e 2021-05-28T01:07:31.209Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=968ac513-e15f-475a-97ba-3b784ff97811 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:31.209Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=968ac513-e15f-475a-97ba-3b784ff97811 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:34.214Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=968ac513-e15f-475a-97ba-3b784ff97811 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:07:34.215Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=968ac513-e15f-475a-97ba-3b784ff97811 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:07:34.215Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=968ac513-e15f-475a-97ba-3b784ff97811 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:34.215Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=968ac513-e15f-475a-97ba-3b784ff97811 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:07:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:07:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:07:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:07:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:07:37.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a88c2cf-ea90-40bf-bcb5-7ac7954ddf77 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:37.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a88c2cf-ea90-40bf-bcb5-7ac7954ddf77 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7af5feca 2021-05-28T01:07:37.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a88c2cf-ea90-40bf-bcb5-7ac7954ddf77 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:37.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a88c2cf-ea90-40bf-bcb5-7ac7954ddf77 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:07:38.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:07:40.399Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=3a88c2cf-ea90-40bf-bcb5-7ac7954ddf77 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:07:40.399Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=3a88c2cf-ea90-40bf-bcb5-7ac7954ddf77 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:07:41.185Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:07:42.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e9a3b9-0d22-4746-8218-01040b698553 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:42.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e9a3b9-0d22-4746-8218-01040b698553 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5fd541d 2021-05-28T01:07:42.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e9a3b9-0d22-4746-8218-01040b698553 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:42.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0e9a3b9-0d22-4746-8218-01040b698553 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:42.675Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b0e9a3b9-0d22-4746-8218-01040b698553 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:42.675Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=b0e9a3b9-0d22-4746-8218-01040b698553 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:07:44.089Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80a65b83-553d-4e39-a837-312eca5e657d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:44.090Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80a65b83-553d-4e39-a837-312eca5e657d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b8be9ab 2021-05-28T01:07:44.090Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80a65b83-553d-4e39-a837-312eca5e657d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:44.090Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80a65b83-553d-4e39-a837-312eca5e657d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:47.095Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=80a65b83-553d-4e39-a837-312eca5e657d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:07:47.095Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=80a65b83-553d-4e39-a837-312eca5e657d ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:07:47.095Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=80a65b83-553d-4e39-a837-312eca5e657d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:47.095Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=80a65b83-553d-4e39-a837-312eca5e657d ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:07:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:07:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:07:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:07:52.116Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:07:52.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0adbe440-efa6-4ef2-b3b9-b716900fcb85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:07:52.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0adbe440-efa6-4ef2-b3b9-b716900fcb85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@696e1751 2021-05-28T01:07:52.480Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0adbe440-efa6-4ef2-b3b9-b716900fcb85 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:07:52.481Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0adbe440-efa6-4ef2-b3b9-b716900fcb85 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:07:53.070Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:07:53.074Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0adbe440-efa6-4ef2-b3b9-b716900fcb85 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:53.074Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=0adbe440-efa6-4ef2-b3b9-b716900fcb85 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:07:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:07:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:07:59.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:08:02.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:08:03.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:03.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@724d7975 2021-05-28T01:08:03.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:03.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:05.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:08:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:08:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:08:06.869Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:08:06.869Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:08:06.869Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:08:06.869Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7b3a3d4e-c74d-4e67-936c-3305fa0d8e3f ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:08:08.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:08:11.069Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:08:12.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c331cf4-de7c-4208-b0f2-10c5afda8190 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:12.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c331cf4-de7c-4208-b0f2-10c5afda8190 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@669f1026 2021-05-28T01:08:12.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c331cf4-de7c-4208-b0f2-10c5afda8190 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:12.369Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c331cf4-de7c-4208-b0f2-10c5afda8190 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:13.072Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=9c331cf4-de7c-4208-b0f2-10c5afda8190 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:08:13.072Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=9c331cf4-de7c-4208-b0f2-10c5afda8190 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:08:14.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea959e7-0028-4dd4-8263-f936a4cebe47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:14.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea959e7-0028-4dd4-8263-f936a4cebe47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e31491f 2021-05-28T01:08:14.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea959e7-0028-4dd4-8263-f936a4cebe47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:14.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea959e7-0028-4dd4-8263-f936a4cebe47 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:17.077Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4ea959e7-0028-4dd4-8263-f936a4cebe47 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:08:17.077Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4ea959e7-0028-4dd4-8263-f936a4cebe47 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:08:17.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a096788-97b5-487d-9007-61c3736ee0c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:17.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a096788-97b5-487d-9007-61c3736ee0c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72e0d7d 2021-05-28T01:08:17.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a096788-97b5-487d-9007-61c3736ee0c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:17.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a096788-97b5-487d-9007-61c3736ee0c0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:18.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527581cb-61a5-4fbd-aaf9-eb301f0e1a43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:18.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527581cb-61a5-4fbd-aaf9-eb301f0e1a43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@192e339e 2021-05-28T01:08:18.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527581cb-61a5-4fbd-aaf9-eb301f0e1a43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:18.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527581cb-61a5-4fbd-aaf9-eb301f0e1a43 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:19.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50ff737-0303-457a-a654-71a17e075635 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:19.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50ff737-0303-457a-a654-71a17e075635 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7428c41f 2021-05-28T01:08:19.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50ff737-0303-457a-a654-71a17e075635 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:19.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50ff737-0303-457a-a654-71a17e075635 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:08:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:08:20.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49194690-d5ce-4eca-80cd-097b452af18a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:20.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49194690-d5ce-4eca-80cd-097b452af18a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60450234 2021-05-28T01:08:20.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49194690-d5ce-4eca-80cd-097b452af18a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:20.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=49194690-d5ce-4eca-80cd-097b452af18a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:22.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24237db1-8a4d-4055-af63-583c9174bcd1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:22.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24237db1-8a4d-4055-af63-583c9174bcd1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1777aea4 2021-05-28T01:08:22.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24237db1-8a4d-4055-af63-583c9174bcd1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:22.562Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24237db1-8a4d-4055-af63-583c9174bcd1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b3015e-8a10-4fc5-803d-c3c80054c714 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b3015e-8a10-4fc5-803d-c3c80054c714 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7faddd17 2021-05-28T01:08:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b3015e-8a10-4fc5-803d-c3c80054c714 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:24.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b3015e-8a10-4fc5-803d-c3c80054c714 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dbd0ee8-f54c-4661-a38f-a98d0f9bf1e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dbd0ee8-f54c-4661-a38f-a98d0f9bf1e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d97e6c1 2021-05-28T01:08:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dbd0ee8-f54c-4661-a38f-a98d0f9bf1e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:25.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dbd0ee8-f54c-4661-a38f-a98d0f9bf1e9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:27.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de72e1e6-78fa-4742-a53c-b62769bd6106 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:27.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de72e1e6-78fa-4742-a53c-b62769bd6106 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58b19e5 2021-05-28T01:08:27.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de72e1e6-78fa-4742-a53c-b62769bd6106 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:27.155Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de72e1e6-78fa-4742-a53c-b62769bd6106 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:28.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f9bc0ad-68d8-4b46-9bfd-4e9f060d5109 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:28.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f9bc0ad-68d8-4b46-9bfd-4e9f060d5109 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f23957e 2021-05-28T01:08:28.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f9bc0ad-68d8-4b46-9bfd-4e9f060d5109 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:28.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f9bc0ad-68d8-4b46-9bfd-4e9f060d5109 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87ba6234-2a38-47b3-9467-d8801370ffc8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87ba6234-2a38-47b3-9467-d8801370ffc8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e77a0e8 2021-05-28T01:08:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87ba6234-2a38-47b3-9467-d8801370ffc8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:29.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87ba6234-2a38-47b3-9467-d8801370ffc8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:30.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b08f413-caab-4036-9c6b-e465eee26fe9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:30.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b08f413-caab-4036-9c6b-e465eee26fe9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c24f809 2021-05-28T01:08:30.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b08f413-caab-4036-9c6b-e465eee26fe9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:30.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b08f413-caab-4036-9c6b-e465eee26fe9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:31.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d2dcc8d-43ef-4335-9ff5-a00123c0f2b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:31.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d2dcc8d-43ef-4335-9ff5-a00123c0f2b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@113d8459 2021-05-28T01:08:31.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d2dcc8d-43ef-4335-9ff5-a00123c0f2b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:31.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d2dcc8d-43ef-4335-9ff5-a00123c0f2b2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:32.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a461013-786a-4d3d-a5ca-0a7c39182fb5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:32.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a461013-786a-4d3d-a5ca-0a7c39182fb5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22066b65 2021-05-28T01:08:32.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a461013-786a-4d3d-a5ca-0a7c39182fb5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:32.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a461013-786a-4d3d-a5ca-0a7c39182fb5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:34.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d20c563a-4b67-4721-90e2-1a15dabbbc24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:34.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d20c563a-4b67-4721-90e2-1a15dabbbc24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2175ef2d 2021-05-28T01:08:34.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d20c563a-4b67-4721-90e2-1a15dabbbc24 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:34.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d20c563a-4b67-4721-90e2-1a15dabbbc24 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:35.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83368835-bac4-4dd6-8f7e-cbcba7b29c82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:35.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83368835-bac4-4dd6-8f7e-cbcba7b29c82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4afaafba 2021-05-28T01:08:35.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83368835-bac4-4dd6-8f7e-cbcba7b29c82 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:35.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83368835-bac4-4dd6-8f7e-cbcba7b29c82 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:08:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:08:36.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f164a7-9efc-4060-a442-7c6fc8e48321 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:36.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f164a7-9efc-4060-a442-7c6fc8e48321 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3718e5f0 2021-05-28T01:08:36.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f164a7-9efc-4060-a442-7c6fc8e48321 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:36.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91f164a7-9efc-4060-a442-7c6fc8e48321 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:08:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:08:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:08:38.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27f54e0b-7e6f-4b0a-958d-5f15106dd725 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:38.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27f54e0b-7e6f-4b0a-958d-5f15106dd725 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26d4fd35 2021-05-28T01:08:38.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27f54e0b-7e6f-4b0a-958d-5f15106dd725 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:38.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27f54e0b-7e6f-4b0a-958d-5f15106dd725 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:39.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87448968-88ca-4eb1-9ae2-1e1964adf27c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:39.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87448968-88ca-4eb1-9ae2-1e1964adf27c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28d66245 2021-05-28T01:08:39.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87448968-88ca-4eb1-9ae2-1e1964adf27c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:39.464Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87448968-88ca-4eb1-9ae2-1e1964adf27c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:40.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6a5ef7-fa1a-4953-9543-26f6b0cb3850 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:40.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6a5ef7-fa1a-4953-9543-26f6b0cb3850 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@374e8f9a 2021-05-28T01:08:40.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6a5ef7-fa1a-4953-9543-26f6b0cb3850 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:40.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a6a5ef7-fa1a-4953-9543-26f6b0cb3850 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:41.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03a0498e-1caf-4419-8339-7a6e58b1c77a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:41.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03a0498e-1caf-4419-8339-7a6e58b1c77a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4789cbe6 2021-05-28T01:08:41.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03a0498e-1caf-4419-8339-7a6e58b1c77a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:41.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03a0498e-1caf-4419-8339-7a6e58b1c77a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:42.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88fbf3f-1be5-45df-8d15-be12aed44546 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:42.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88fbf3f-1be5-45df-8d15-be12aed44546 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46b5ab77 2021-05-28T01:08:42.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88fbf3f-1be5-45df-8d15-be12aed44546 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:42.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b88fbf3f-1be5-45df-8d15-be12aed44546 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:44.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94c020bb-1537-40fe-8581-0c0f21c9695b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:44.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94c020bb-1537-40fe-8581-0c0f21c9695b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d6a2dae 2021-05-28T01:08:44.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94c020bb-1537-40fe-8581-0c0f21c9695b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:44.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94c020bb-1537-40fe-8581-0c0f21c9695b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:46.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5f3a63-386e-4d7a-872b-108cd54fbf59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:46.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5f3a63-386e-4d7a-872b-108cd54fbf59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e9b4bc6 2021-05-28T01:08:46.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5f3a63-386e-4d7a-872b-108cd54fbf59 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:46.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e5f3a63-386e-4d7a-872b-108cd54fbf59 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:49.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c853bd19-5eeb-41e4-80c0-cce8c19f3a6f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:49.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c853bd19-5eeb-41e4-80c0-cce8c19f3a6f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17447508 2021-05-28T01:08:49.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c853bd19-5eeb-41e4-80c0-cce8c19f3a6f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:49.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c853bd19-5eeb-41e4-80c0-cce8c19f3a6f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:08:50.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:08:52.144Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:08:52.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0f74269-9c08-4b44-a499-8ba9f15b5465 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:52.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0f74269-9c08-4b44-a499-8ba9f15b5465 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c419a84 2021-05-28T01:08:52.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0f74269-9c08-4b44-a499-8ba9f15b5465 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:52.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0f74269-9c08-4b44-a499-8ba9f15b5465 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:55.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bd691d2-89e2-4362-89e4-fda04928d69b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:55.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bd691d2-89e2-4362-89e4-fda04928d69b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@704acfff 2021-05-28T01:08:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bd691d2-89e2-4362-89e4-fda04928d69b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:55.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bd691d2-89e2-4362-89e4-fda04928d69b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:08:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df731f7-6085-4447-8bec-d233bbcde393 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:08:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df731f7-6085-4447-8bec-d233bbcde393 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34cb7f4b 2021-05-28T01:08:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df731f7-6085-4447-8bec-d233bbcde393 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:08:58.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8df731f7-6085-4447-8bec-d233bbcde393 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78955cd5-dfe3-4f7a-9028-53b9a80bc574 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78955cd5-dfe3-4f7a-9028-53b9a80bc574 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@93a0ef 2021-05-28T01:09:01.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78955cd5-dfe3-4f7a-9028-53b9a80bc574 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:01.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78955cd5-dfe3-4f7a-9028-53b9a80bc574 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:04.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed050c57-67b2-4f09-be1a-8786209e0886 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:04.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed050c57-67b2-4f09-be1a-8786209e0886 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70769fc3 2021-05-28T01:09:04.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed050c57-67b2-4f09-be1a-8786209e0886 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:04.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed050c57-67b2-4f09-be1a-8786209e0886 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:09:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:09:07.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74181f8d-2e26-4655-8602-aa1833b245de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:07.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74181f8d-2e26-4655-8602-aa1833b245de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56658305 2021-05-28T01:09:07.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74181f8d-2e26-4655-8602-aa1833b245de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:07.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74181f8d-2e26-4655-8602-aa1833b245de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:10.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb5bd54e-aa74-448c-bebd-f051d3e6a853 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:10.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb5bd54e-aa74-448c-bebd-f051d3e6a853 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@393b8fa5 2021-05-28T01:09:10.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb5bd54e-aa74-448c-bebd-f051d3e6a853 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:10.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb5bd54e-aa74-448c-bebd-f051d3e6a853 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3600f09a-1c69-490e-8b6e-fff8d1f2f736 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3600f09a-1c69-490e-8b6e-fff8d1f2f736 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24b2d4b5 2021-05-28T01:09:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3600f09a-1c69-490e-8b6e-fff8d1f2f736 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:13.979Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3600f09a-1c69-490e-8b6e-fff8d1f2f736 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:16.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e03a95c1-5c24-402d-8c16-1bbdd7543767 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:16.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e03a95c1-5c24-402d-8c16-1bbdd7543767 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@607ecf67 2021-05-28T01:09:16.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e03a95c1-5c24-402d-8c16-1bbdd7543767 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:16.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e03a95c1-5c24-402d-8c16-1bbdd7543767 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:19.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1e48500-7335-49d7-8faf-49c78a85f240 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:19.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1e48500-7335-49d7-8faf-49c78a85f240 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1af905c6 2021-05-28T01:09:19.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1e48500-7335-49d7-8faf-49c78a85f240 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:19.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1e48500-7335-49d7-8faf-49c78a85f240 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:09:20.517Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:09:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea4b6a06-f35c-445c-b38b-5efd24bd6e30 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea4b6a06-f35c-445c-b38b-5efd24bd6e30 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fdae79b 2021-05-28T01:09:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea4b6a06-f35c-445c-b38b-5efd24bd6e30 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:22.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea4b6a06-f35c-445c-b38b-5efd24bd6e30 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:26.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6585805f-12dd-4f73-a562-d3ce37868c13 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:26.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6585805f-12dd-4f73-a562-d3ce37868c13 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4dff83a9 2021-05-28T01:09:26.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6585805f-12dd-4f73-a562-d3ce37868c13 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:26.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6585805f-12dd-4f73-a562-d3ce37868c13 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:28.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54929863-3166-45f8-98c8-b839315adf51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:28.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54929863-3166-45f8-98c8-b839315adf51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5bec7886 2021-05-28T01:09:28.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54929863-3166-45f8-98c8-b839315adf51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:28.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54929863-3166-45f8-98c8-b839315adf51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76dea28c-f109-4704-b3cc-75b162d64f62 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76dea28c-f109-4704-b3cc-75b162d64f62 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9e99108 2021-05-28T01:09:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76dea28c-f109-4704-b3cc-75b162d64f62 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:31.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76dea28c-f109-4704-b3cc-75b162d64f62 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13239843-d202-4db4-9c9d-840f85a2816e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13239843-d202-4db4-9c9d-840f85a2816e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21a75991 2021-05-28T01:09:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13239843-d202-4db4-9c9d-840f85a2816e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:34.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=13239843-d202-4db4-9c9d-840f85a2816e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:09:35.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:09:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:09:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:09:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:09:38.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe79713a-aca1-4cd6-b8df-40ef586726be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:38.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe79713a-aca1-4cd6-b8df-40ef586726be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10f474c7 2021-05-28T01:09:38.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe79713a-aca1-4cd6-b8df-40ef586726be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:38.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe79713a-aca1-4cd6-b8df-40ef586726be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:41.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ce1ca55-ceb8-4f45-80a6-92e01e7af49a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:41.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ce1ca55-ceb8-4f45-80a6-92e01e7af49a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28e40617 2021-05-28T01:09:41.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ce1ca55-ceb8-4f45-80a6-92e01e7af49a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:41.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ce1ca55-ceb8-4f45-80a6-92e01e7af49a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:44.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c503bd29-99f6-4384-927f-1ace90255037 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:44.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c503bd29-99f6-4384-927f-1ace90255037 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d40a3d2 2021-05-28T01:09:44.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c503bd29-99f6-4384-927f-1ace90255037 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:44.368Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c503bd29-99f6-4384-927f-1ace90255037 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db714a03-1810-43e5-886c-a816bd18c333 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:46.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db714a03-1810-43e5-886c-a816bd18c333 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7630143a 2021-05-28T01:09:46.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db714a03-1810-43e5-886c-a816bd18c333 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:46.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db714a03-1810-43e5-886c-a816bd18c333 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:49.808Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee264d2e-89d6-42d2-ada1-5ce523d1bba5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:49.809Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee264d2e-89d6-42d2-ada1-5ce523d1bba5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45510477 2021-05-28T01:09:49.809Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee264d2e-89d6-42d2-ada1-5ce523d1bba5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:49.809Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee264d2e-89d6-42d2-ada1-5ce523d1bba5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:09:50.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:09:52.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:09:52.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2689e800-00fc-4c7c-aba2-8af571acd362 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:52.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2689e800-00fc-4c7c-aba2-8af571acd362 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d0d6674 2021-05-28T01:09:52.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2689e800-00fc-4c7c-aba2-8af571acd362 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:52.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2689e800-00fc-4c7c-aba2-8af571acd362 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:09:56.052Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c454ed7-199f-4b82-8102-54757771772f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:09:56.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c454ed7-199f-4b82-8102-54757771772f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@253b1234 2021-05-28T01:09:56.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c454ed7-199f-4b82-8102-54757771772f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:09:56.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c454ed7-199f-4b82-8102-54757771772f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:02.475Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3182964-70a9-47dc-bec4-478e6162175e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:02.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3182964-70a9-47dc-bec4-478e6162175e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@535c536d 2021-05-28T01:10:02.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3182964-70a9-47dc-bec4-478e6162175e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:02.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3182964-70a9-47dc-bec4-478e6162175e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:04.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112625eb-1d29-4c31-aeea-3da6d4ea1329 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:04.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112625eb-1d29-4c31-aeea-3da6d4ea1329 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ee397cd 2021-05-28T01:10:04.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112625eb-1d29-4c31-aeea-3da6d4ea1329 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:04.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=112625eb-1d29-4c31-aeea-3da6d4ea1329 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:10:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:10:05.988Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba219bbc-d1a7-43ac-93dc-bac395f4220e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:05.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba219bbc-d1a7-43ac-93dc-bac395f4220e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@387b7ca3 2021-05-28T01:10:05.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba219bbc-d1a7-43ac-93dc-bac395f4220e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:05.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba219bbc-d1a7-43ac-93dc-bac395f4220e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:08.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f70b6f0c-aebd-43a7-820d-87fc0456eb73 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:08.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f70b6f0c-aebd-43a7-820d-87fc0456eb73 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45ac632a 2021-05-28T01:10:08.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f70b6f0c-aebd-43a7-820d-87fc0456eb73 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:08.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f70b6f0c-aebd-43a7-820d-87fc0456eb73 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:12.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1d85b77-a800-4729-b860-ef5bfbd9e1e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:12.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1d85b77-a800-4729-b860-ef5bfbd9e1e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74da97c9 2021-05-28T01:10:12.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1d85b77-a800-4729-b860-ef5bfbd9e1e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:12.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b1d85b77-a800-4729-b860-ef5bfbd9e1e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:14.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0364be0b-76c2-468a-bb8a-1666f38bae37 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:14.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0364be0b-76c2-468a-bb8a-1666f38bae37 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76052b60 2021-05-28T01:10:14.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0364be0b-76c2-468a-bb8a-1666f38bae37 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:14.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0364be0b-76c2-468a-bb8a-1666f38bae37 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3fe6257-329a-4364-8923-755374eb199c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3fe6257-329a-4364-8923-755374eb199c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1578df67 2021-05-28T01:10:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3fe6257-329a-4364-8923-755374eb199c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:16.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3fe6257-329a-4364-8923-755374eb199c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:20.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e825fd41-6cba-4558-80da-cdd8e22d5a0b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:20.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e825fd41-6cba-4558-80da-cdd8e22d5a0b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33eb4bd1 2021-05-28T01:10:20.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e825fd41-6cba-4558-80da-cdd8e22d5a0b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:20.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e825fd41-6cba-4558-80da-cdd8e22d5a0b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:10:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:10:22.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d8603d-9802-4e32-8669-8954351a6691 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:22.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d8603d-9802-4e32-8669-8954351a6691 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13afe6ba 2021-05-28T01:10:22.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d8603d-9802-4e32-8669-8954351a6691 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:22.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d8603d-9802-4e32-8669-8954351a6691 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:28.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e8e718b-ad8e-4210-b7f9-473ca6228260 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:28.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e8e718b-ad8e-4210-b7f9-473ca6228260 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4cae8184 2021-05-28T01:10:28.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e8e718b-ad8e-4210-b7f9-473ca6228260 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:28.470Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e8e718b-ad8e-4210-b7f9-473ca6228260 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:29.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0305264-bf7e-4b7d-8a24-cf5a340876bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:29.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0305264-bf7e-4b7d-8a24-cf5a340876bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f7efc91 2021-05-28T01:10:29.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0305264-bf7e-4b7d-8a24-cf5a340876bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:29.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0305264-bf7e-4b7d-8a24-cf5a340876bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:32.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7fa52be-f839-4be2-86bf-cd454b03a52e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:32.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7fa52be-f839-4be2-86bf-cd454b03a52e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7622996c 2021-05-28T01:10:32.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7fa52be-f839-4be2-86bf-cd454b03a52e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:32.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7fa52be-f839-4be2-86bf-cd454b03a52e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:35.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:10:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:10:35.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0acede0-3760-464e-9519-b2ab9a0d6b3d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:35.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0acede0-3760-464e-9519-b2ab9a0d6b3d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a543b24 2021-05-28T01:10:35.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0acede0-3760-464e-9519-b2ab9a0d6b3d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:35.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0acede0-3760-464e-9519-b2ab9a0d6b3d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:10:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:10:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:10:39.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7d0a0a1-cf94-4d31-acc0-14771fedb4b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:39.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7d0a0a1-cf94-4d31-acc0-14771fedb4b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@224e2b4b 2021-05-28T01:10:39.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7d0a0a1-cf94-4d31-acc0-14771fedb4b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:39.173Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7d0a0a1-cf94-4d31-acc0-14771fedb4b8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41d8834b-efd5-431b-a58a-9ce1f3aa30be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41d8834b-efd5-431b-a58a-9ce1f3aa30be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4abcc1e7 2021-05-28T01:10:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41d8834b-efd5-431b-a58a-9ce1f3aa30be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41d8834b-efd5-431b-a58a-9ce1f3aa30be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:43.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5092119a-8cb5-4ef1-a82b-2e85283422e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:43.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5092119a-8cb5-4ef1-a82b-2e85283422e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2837d825 2021-05-28T01:10:43.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5092119a-8cb5-4ef1-a82b-2e85283422e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:43.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5092119a-8cb5-4ef1-a82b-2e85283422e0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:46.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbf20ffa-7708-42c7-960a-36429ed3a332 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:46.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbf20ffa-7708-42c7-960a-36429ed3a332 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13814b1c 2021-05-28T01:10:46.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbf20ffa-7708-42c7-960a-36429ed3a332 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:46.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbf20ffa-7708-42c7-960a-36429ed3a332 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:49.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dab9909c-feb7-423f-864d-f2a22a8677ae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:49.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dab9909c-feb7-423f-864d-f2a22a8677ae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40c62e54 2021-05-28T01:10:49.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dab9909c-feb7-423f-864d-f2a22a8677ae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:49.880Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dab9909c-feb7-423f-864d-f2a22a8677ae PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:10:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:10:52.219Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:10:52.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cb8408d-e111-4204-88cd-0b5ad95c2cdc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:52.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cb8408d-e111-4204-88cd-0b5ad95c2cdc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cd96c52 2021-05-28T01:10:52.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cb8408d-e111-4204-88cd-0b5ad95c2cdc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:52.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cb8408d-e111-4204-88cd-0b5ad95c2cdc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:10:56.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:10:58.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52219fcd-32ff-4189-957f-e578cb4e7ade PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:58.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52219fcd-32ff-4189-957f-e578cb4e7ade PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ab9771c 2021-05-28T01:10:58.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52219fcd-32ff-4189-957f-e578cb4e7ade PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:58.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52219fcd-32ff-4189-957f-e578cb4e7ade PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:10:58.577Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:10:59.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb7cf28-89ee-495d-8260-5691a816508d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:10:59.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb7cf28-89ee-495d-8260-5691a816508d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63790afc 2021-05-28T01:10:59.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb7cf28-89ee-495d-8260-5691a816508d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:10:59.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb7cf28-89ee-495d-8260-5691a816508d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:00.483Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=abb7cf28-89ee-495d-8260-5691a816508d ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:11:00.483Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=abb7cf28-89ee-495d-8260-5691a816508d ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:11:01.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7a4b38e-4ab9-488f-9d39-072f36cfbb3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:01.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7a4b38e-4ab9-488f-9d39-072f36cfbb3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12bc3c49 2021-05-28T01:11:01.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7a4b38e-4ab9-488f-9d39-072f36cfbb3f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:01.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7a4b38e-4ab9-488f-9d39-072f36cfbb3f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:02.680Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d7a4b38e-4ab9-488f-9d39-072f36cfbb3f ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:11:02.680Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d7a4b38e-4ab9-488f-9d39-072f36cfbb3f ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f264913-fac9-4a23-b63c-88ab2bb54cab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f264913-fac9-4a23-b63c-88ab2bb54cab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@352079b0 2021-05-28T01:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f264913-fac9-4a23-b63c-88ab2bb54cab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:05.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f264913-fac9-4a23-b63c-88ab2bb54cab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:11:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:11:07.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f031d9-a464-408e-a989-b3fda4f07d50 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:07.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f031d9-a464-408e-a989-b3fda4f07d50 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@490e99f 2021-05-28T01:11:07.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f031d9-a464-408e-a989-b3fda4f07d50 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:07.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=38f031d9-a464-408e-a989-b3fda4f07d50 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:11:11.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:11:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e763b43-1a77-4499-88c5-4d1aec354b8e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e763b43-1a77-4499-88c5-4d1aec354b8e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f153d1b 2021-05-28T01:11:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e763b43-1a77-4499-88c5-4d1aec354b8e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e763b43-1a77-4499-88c5-4d1aec354b8e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:13.660Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:11:14.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a9c5d1d-e80e-486f-9441-85cba95416b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:14.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a9c5d1d-e80e-486f-9441-85cba95416b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cb812c4 2021-05-28T01:11:14.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a9c5d1d-e80e-486f-9441-85cba95416b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:14.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a9c5d1d-e80e-486f-9441-85cba95416b1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:15.559Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=1a9c5d1d-e80e-486f-9441-85cba95416b1 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:11:15.559Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=1a9c5d1d-e80e-486f-9441-85cba95416b1 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:11:16.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3085f4b7-db57-4b67-a51d-74f2b458b6f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:16.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3085f4b7-db57-4b67-a51d-74f2b458b6f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a631d1a 2021-05-28T01:11:16.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3085f4b7-db57-4b67-a51d-74f2b458b6f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:16.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3085f4b7-db57-4b67-a51d-74f2b458b6f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:17.876Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3085f4b7-db57-4b67-a51d-74f2b458b6f0 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:11:17.876Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3085f4b7-db57-4b67-a51d-74f2b458b6f0 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:11:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b69590-2fb5-41f9-b7d7-c3141c89c617 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b69590-2fb5-41f9-b7d7-c3141c89c617 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69bccdd8 2021-05-28T01:11:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b69590-2fb5-41f9-b7d7-c3141c89c617 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:19.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86b69590-2fb5-41f9-b7d7-c3141c89c617 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:11:20.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:11:22.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffd9985a-5cd8-4a56-9847-c5f70166968d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:22.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffd9985a-5cd8-4a56-9847-c5f70166968d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fc7c811 2021-05-28T01:11:22.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffd9985a-5cd8-4a56-9847-c5f70166968d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:22.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffd9985a-5cd8-4a56-9847-c5f70166968d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:25.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad319950-127d-4547-9f2d-ab173bc04083 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:25.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad319950-127d-4547-9f2d-ab173bc04083 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73f5d91f 2021-05-28T01:11:25.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad319950-127d-4547-9f2d-ab173bc04083 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:25.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad319950-127d-4547-9f2d-ab173bc04083 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:29.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cfa12ed5-5767-4f3e-91bd-a2450b20dde4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:29.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cfa12ed5-5767-4f3e-91bd-a2450b20dde4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18c91b6 2021-05-28T01:11:29.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cfa12ed5-5767-4f3e-91bd-a2450b20dde4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:29.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cfa12ed5-5767-4f3e-91bd-a2450b20dde4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:31.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4885521-e48b-4a55-871e-d88cd17975da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:31.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4885521-e48b-4a55-871e-d88cd17975da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e515e52 2021-05-28T01:11:31.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4885521-e48b-4a55-871e-d88cd17975da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:31.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4885521-e48b-4a55-871e-d88cd17975da PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:34.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d004eacd-5822-43cb-9ee2-8782dd666622 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:34.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d004eacd-5822-43cb-9ee2-8782dd666622 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f8788c9 2021-05-28T01:11:34.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d004eacd-5822-43cb-9ee2-8782dd666622 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:34.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d004eacd-5822-43cb-9ee2-8782dd666622 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:11:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:11:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:11:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:11:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:11:37.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66eaf6ee-f112-438e-96df-3ca261e94149 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:37.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66eaf6ee-f112-438e-96df-3ca261e94149 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2678f07 2021-05-28T01:11:37.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66eaf6ee-f112-438e-96df-3ca261e94149 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:37.563Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66eaf6ee-f112-438e-96df-3ca261e94149 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:40.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8c9876-f388-4008-b9d7-db2d2fa2277c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:40.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8c9876-f388-4008-b9d7-db2d2fa2277c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6783afdc 2021-05-28T01:11:40.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8c9876-f388-4008-b9d7-db2d2fa2277c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:40.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd8c9876-f388-4008-b9d7-db2d2fa2277c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:43.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ae0219e-2d43-4ac2-9b16-af04c3d06204 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:43.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ae0219e-2d43-4ac2-9b16-af04c3d06204 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6324deb0 2021-05-28T01:11:43.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ae0219e-2d43-4ac2-9b16-af04c3d06204 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:43.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5ae0219e-2d43-4ac2-9b16-af04c3d06204 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:47.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=843db585-d472-4394-909e-a4ae0a599234 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:47.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=843db585-d472-4394-909e-a4ae0a599234 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c3dd 2021-05-28T01:11:47.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=843db585-d472-4394-909e-a4ae0a599234 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:47.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=843db585-d472-4394-909e-a4ae0a599234 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:49.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b65a83c-3276-4275-a536-5ff2f447115e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:49.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b65a83c-3276-4275-a536-5ff2f447115e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e276b6d 2021-05-28T01:11:49.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b65a83c-3276-4275-a536-5ff2f447115e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:49.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b65a83c-3276-4275-a536-5ff2f447115e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:11:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:11:52.124Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:11:52.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d4d1c88-642f-45ac-9894-a32accd1efff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:52.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d4d1c88-642f-45ac-9894-a32accd1efff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4420de3a 2021-05-28T01:11:52.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d4d1c88-642f-45ac-9894-a32accd1efff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:52.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d4d1c88-642f-45ac-9894-a32accd1efff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:56.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4702ca19-32b6-4171-91f3-7ce29f0dbf5a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:56.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4702ca19-32b6-4171-91f3-7ce29f0dbf5a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1344950 2021-05-28T01:11:56.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4702ca19-32b6-4171-91f3-7ce29f0dbf5a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:56.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4702ca19-32b6-4171-91f3-7ce29f0dbf5a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:11:58.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cafd6df7-6101-4810-b45c-2777716118a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:11:58.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cafd6df7-6101-4810-b45c-2777716118a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@371a7a80 2021-05-28T01:11:58.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cafd6df7-6101-4810-b45c-2777716118a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:11:58.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cafd6df7-6101-4810-b45c-2777716118a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:01.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6e9c4a5-e69f-48ca-94bd-d22fd2d8b5c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:01.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6e9c4a5-e69f-48ca-94bd-d22fd2d8b5c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3263030d 2021-05-28T01:12:01.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6e9c4a5-e69f-48ca-94bd-d22fd2d8b5c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:01.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6e9c4a5-e69f-48ca-94bd-d22fd2d8b5c4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:05.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=074545f8-ecea-4afc-9377-e207698f7645 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:05.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=074545f8-ecea-4afc-9377-e207698f7645 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7070da75 2021-05-28T01:12:05.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=074545f8-ecea-4afc-9377-e207698f7645 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:05.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=074545f8-ecea-4afc-9377-e207698f7645 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:12:05.518Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:12:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811e605d-8b79-42b5-82c8-2ed98b0b1df2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811e605d-8b79-42b5-82c8-2ed98b0b1df2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53ad73e2 2021-05-28T01:12:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811e605d-8b79-42b5-82c8-2ed98b0b1df2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811e605d-8b79-42b5-82c8-2ed98b0b1df2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02fxaticnn09u5plvskzomp18263 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node03ivsw1zldd4kghevby7kynf0291 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kpx2migeih3n1v4nuh3xb28jf283 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01a2qda91p94coakwh1ygf9idj254 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0qjv2ffaa50vb1tx6c9lz476ng255 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ay39hyecyz01rf95uvk65e5f278 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010alvxx8mtnxrr5eneu87bn27289 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tdcou7v9lqfkysqnsmht7jpk292 2021-05-28T01:12:08.160Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01r6prp2rmdiz41f2upu5nq3s3322 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0s3q6ejybww7k13x3itnsu4wk6294 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0w8bnqtxhgv4r1gaddlemv77w1300 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gnawfn92ne47fnwylu87p5o3321 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06krkfyekv4kh1c41ufz952f5e326 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0183ykvxljp0q3ev42255tg269295 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0a7pqky11wehg1slvtrw0o0rya299 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0hmzdxhkaqeyd1xoea0krs93nr276 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0a2xa17b3vjhlakpv76jg57su281 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0n3wjlz6dtxkpe12i88vwc0w284 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node015qs4ibyav9io1vb47v0ieo88d311 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node04nbowvo6sjeebddntb4xdiq1314 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01b6tfqih1k1imlaz5dpsjuioz309 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01rai7foqfm3ppy48j1ijn6z0k306 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06tjxmkrreja71l1y0yzxixmw0272 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0rflj4hccxaw61xf40uxnb2jsm327 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0bcjj5ukvhsnxzkdrqvz16g42308 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0qytsnnjttv2slaefru5lhsbu313 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ceecsb9vjxv51q9swb2ngn07e266 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kds9b2855avquw95fpkomw2k275 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010okpqb15jum3tc5gpmtvwmvq317 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0l0cupl23b9vp5zvjz9g8eic1305 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xg2gij3v6lwp6073rhjuw706320 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011s4ktyhin7m8evv2wzttj5z0262 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node019mm4zqd8v8dm1w8vtczawn2u4285 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0177992tn0sz5ugbnnkakpmany259 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0fv59fku6wgmq1i44n31m9a173328 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01q2ao8mb2x6lr1355r14fiy8i5318 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xmjd0uxm4dtf18upgw0r8wlic324 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06oamuftglcer19whmbf4cpqr3302 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0k0b2e278o2luhm939u2wtqwj319 2021-05-28T01:12:08.161Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01979zi50s8yau1nyfrddgb28if257 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01esn68hgzijfvx9tjz5z5p2bz260 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0hryi58knbq1pe830eutaa41a312 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0t62np68ujwxa6iptq2zyuuop296 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yhht1pwronp6168uofstkjcsx264 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tjryxu6anj0i2hpxzwwy91u4323 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01prwtiu9yient16famr33egabb315 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0p2jbejzthleuvcnx2i9ixtd3253 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0b1kp5hl54mhnjaiabdns4m1i265 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0banu1r1rnj1z15wm618s42yg7304 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node017s6deqmc20dd1bh55vqcpnbno256 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01muk9czie58zv8cf46vtuiiww301 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06kzlxv6hmi6y1ppuoq5wl6rw1288 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0183x7yvk3dqhk8rhlk7u3qmat280 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ji61ytzpt41v1g6gn6zczqe3v270 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01420h5e2c5ye02bp9peflenj6293 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0oc2o69fl3g881nmei7lsda5hp273 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01u5jlwnym7bk8na6o1innxpdn269 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01dp1jqcpz26yefeb28rmw421a303 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node08df6bgwfc4w015dqvb220r7rm290 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0f97a5ayc5kkhn3ly61cy1ogp279 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01e7z1pu5s1hm4tytffa9ghi2k271 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node07lq71842h95u2gm0ire64agf277 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0m9u9s8s3l6qm1om30xuddzziq261 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010tucu3hiiuzbx78r9d4346yn316 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01aqu4xakgoockt9brb7um0u8p274 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qw3akcy5cbva5chop0pcwak4310 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0pveslx7r83l31nkhkotd3cn2q258 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0119taepe1500618lp9pedktjhy267 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tnmddrulwzcx6n1hnvd0cqx6329 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qmiabuvsik9y1e0sywy9soyiw307 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09sga2sz8g6zr1hpr73e5fw7ix287 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01jy1f3b1b7qsv7uur77v9e70u325 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010tcbhnwvkofsbdd6aqj2ypgp286 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node08fy49t9l5qyv1plybfncscjg3298 2021-05-28T01:12:08.162Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0unvr0eqyg9it1llxerfersrwl268 2021-05-28T01:12:08.163Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01b5icdlvl975116s2cdz68j9t8282 2021-05-28T01:12:08.163Z [Scheduler-712609105-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yhu8l4ls49apq8kcvoglcmad297 2021-05-28T01:12:11.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34a2eb49-1976-4a9a-924c-c12787e0b18c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:11.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34a2eb49-1976-4a9a-924c-c12787e0b18c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3787cd6d 2021-05-28T01:12:11.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34a2eb49-1976-4a9a-924c-c12787e0b18c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:11.394Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34a2eb49-1976-4a9a-924c-c12787e0b18c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:14.566Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=236d1e66-e13f-49a4-a658-9f9358d616f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:14.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=236d1e66-e13f-49a4-a658-9f9358d616f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dd6cbfb 2021-05-28T01:12:14.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=236d1e66-e13f-49a4-a658-9f9358d616f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:14.567Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=236d1e66-e13f-49a4-a658-9f9358d616f2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6f881d-1fdc-4f9f-997c-2a7e33871530 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6f881d-1fdc-4f9f-997c-2a7e33871530 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b030946 2021-05-28T01:12:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6f881d-1fdc-4f9f-997c-2a7e33871530 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae6f881d-1fdc-4f9f-997c-2a7e33871530 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:20.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91bac790-6be3-4459-819d-ddc0c6b2d5e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:20.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91bac790-6be3-4459-819d-ddc0c6b2d5e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@530945a7 2021-05-28T01:12:20.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91bac790-6be3-4459-819d-ddc0c6b2d5e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:20.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91bac790-6be3-4459-819d-ddc0c6b2d5e9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:12:20.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:12:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0431dbf5-1cb7-48fe-9c43-37a0c0d01001 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0431dbf5-1cb7-48fe-9c43-37a0c0d01001 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a6c160e 2021-05-28T01:12:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0431dbf5-1cb7-48fe-9c43-37a0c0d01001 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:22.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0431dbf5-1cb7-48fe-9c43-37a0c0d01001 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:25.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4eaf85-addb-482f-b4db-b358b4ab3656 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:25.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4eaf85-addb-482f-b4db-b358b4ab3656 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71ce87ab 2021-05-28T01:12:25.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4eaf85-addb-482f-b4db-b358b4ab3656 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:25.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f4eaf85-addb-482f-b4db-b358b4ab3656 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4abb2bd1-7232-42ec-9719-0e6b676948e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4abb2bd1-7232-42ec-9719-0e6b676948e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@274f9ed 2021-05-28T01:12:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4abb2bd1-7232-42ec-9719-0e6b676948e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:28.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4abb2bd1-7232-42ec-9719-0e6b676948e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:31.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039a62a9-41e8-4880-95d7-12f4829817df PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:31.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039a62a9-41e8-4880-95d7-12f4829817df PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1bc9b079 2021-05-28T01:12:31.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039a62a9-41e8-4880-95d7-12f4829817df PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:31.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=039a62a9-41e8-4880-95d7-12f4829817df PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:34.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0741961-dfcf-4f30-a7dc-5e43294ec7f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:34.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0741961-dfcf-4f30-a7dc-5e43294ec7f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@212f0968 2021-05-28T01:12:34.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0741961-dfcf-4f30-a7dc-5e43294ec7f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:34.999Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0741961-dfcf-4f30-a7dc-5e43294ec7f8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:35.508Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:12:35.517Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:12:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:12:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:12:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:12:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1de899b-c09d-4275-99c8-979751807f0c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1de899b-c09d-4275-99c8-979751807f0c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24d92139 2021-05-28T01:12:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1de899b-c09d-4275-99c8-979751807f0c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1de899b-c09d-4275-99c8-979751807f0c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:40.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47a8ab16-26e1-4e2b-9462-8503a1cbccda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:40.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47a8ab16-26e1-4e2b-9462-8503a1cbccda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1470fde 2021-05-28T01:12:40.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47a8ab16-26e1-4e2b-9462-8503a1cbccda PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:40.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47a8ab16-26e1-4e2b-9462-8503a1cbccda PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ee36f02-b7c0-4f22-b60e-285767aeea74 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ee36f02-b7c0-4f22-b60e-285767aeea74 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@575b3df7 2021-05-28T01:12:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ee36f02-b7c0-4f22-b60e-285767aeea74 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ee36f02-b7c0-4f22-b60e-285767aeea74 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:47.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d36faa78-7494-496b-b871-00cc6e3cc6b5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:47.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d36faa78-7494-496b-b871-00cc6e3cc6b5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e661a3d 2021-05-28T01:12:47.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d36faa78-7494-496b-b871-00cc6e3cc6b5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:47.076Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d36faa78-7494-496b-b871-00cc6e3cc6b5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab28439e-d194-499c-a8fe-3991644bf9f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab28439e-d194-499c-a8fe-3991644bf9f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@635b2221 2021-05-28T01:12:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab28439e-d194-499c-a8fe-3991644bf9f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:49.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab28439e-d194-499c-a8fe-3991644bf9f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:12:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:12:52.225Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:12:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2375d799-d41d-4c10-b803-b35941d06fa4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2375d799-d41d-4c10-b803-b35941d06fa4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a5bf86b 2021-05-28T01:12:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2375d799-d41d-4c10-b803-b35941d06fa4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2375d799-d41d-4c10-b803-b35941d06fa4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fec1818-b37d-4a98-86c4-1fdb557dbfbd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fec1818-b37d-4a98-86c4-1fdb557dbfbd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5461d252 2021-05-28T01:12:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fec1818-b37d-4a98-86c4-1fdb557dbfbd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fec1818-b37d-4a98-86c4-1fdb557dbfbd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:12:59.276Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bcb0d6-3c9c-407d-b55c-df687c9f8748 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:12:59.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bcb0d6-3c9c-407d-b55c-df687c9f8748 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ae89d11 2021-05-28T01:12:59.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bcb0d6-3c9c-407d-b55c-df687c9f8748 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:12:59.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bcb0d6-3c9c-407d-b55c-df687c9f8748 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:02.366Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27af84a0-a33b-4b40-8c0c-b63765ffbde5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:02.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27af84a0-a33b-4b40-8c0c-b63765ffbde5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5098f047 2021-05-28T01:13:02.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27af84a0-a33b-4b40-8c0c-b63765ffbde5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:02.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27af84a0-a33b-4b40-8c0c-b63765ffbde5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:05.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bf5ce9b-231c-45a9-b3bf-87ee8ba9527a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:05.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bf5ce9b-231c-45a9-b3bf-87ee8ba9527a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5505c29f 2021-05-28T01:13:05.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bf5ce9b-231c-45a9-b3bf-87ee8ba9527a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:05.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bf5ce9b-231c-45a9-b3bf-87ee8ba9527a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:13:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:13:09.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a408d80c-c1a9-416f-a6dd-6a63cae51782 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:09.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a408d80c-c1a9-416f-a6dd-6a63cae51782 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fac792f 2021-05-28T01:13:09.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a408d80c-c1a9-416f-a6dd-6a63cae51782 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:09.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a408d80c-c1a9-416f-a6dd-6a63cae51782 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2d348f8-3f63-40c9-9962-9a2fc3b6f55a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2d348f8-3f63-40c9-9962-9a2fc3b6f55a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21826dde 2021-05-28T01:13:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2d348f8-3f63-40c9-9962-9a2fc3b6f55a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:12.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2d348f8-3f63-40c9-9962-9a2fc3b6f55a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd26111e-7edb-403f-b4e9-ef8f91020c79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd26111e-7edb-403f-b4e9-ef8f91020c79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@360fbb61 2021-05-28T01:13:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd26111e-7edb-403f-b4e9-ef8f91020c79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:13.969Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd26111e-7edb-403f-b4e9-ef8f91020c79 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:17.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25283f39-2301-46cc-a79c-a24789070037 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:17.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25283f39-2301-46cc-a79c-a24789070037 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d658605 2021-05-28T01:13:17.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25283f39-2301-46cc-a79c-a24789070037 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:17.359Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25283f39-2301-46cc-a79c-a24789070037 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb7b28f-271e-42bd-ae5e-c612b7848f95 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:19.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb7b28f-271e-42bd-ae5e-c612b7848f95 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@216cfb87 2021-05-28T01:13:19.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb7b28f-271e-42bd-ae5e-c612b7848f95 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:19.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dfb7b28f-271e-42bd-ae5e-c612b7848f95 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:13:20.515Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:13:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52a353d9-eea8-42b4-b1d0-4351d03084d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52a353d9-eea8-42b4-b1d0-4351d03084d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13b29ee9 2021-05-28T01:13:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52a353d9-eea8-42b4-b1d0-4351d03084d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:22.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52a353d9-eea8-42b4-b1d0-4351d03084d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:25.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8a002a6-da66-473a-94cc-e8e9c40c1c0a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:25.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8a002a6-da66-473a-94cc-e8e9c40c1c0a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58adeb4d 2021-05-28T01:13:25.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8a002a6-da66-473a-94cc-e8e9c40c1c0a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:25.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8a002a6-da66-473a-94cc-e8e9c40c1c0a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:29.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e2f8c4-613f-4141-bded-5c3e96f53ead PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:29.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e2f8c4-613f-4141-bded-5c3e96f53ead PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39191bf3 2021-05-28T01:13:29.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e2f8c4-613f-4141-bded-5c3e96f53ead PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:29.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e2f8c4-613f-4141-bded-5c3e96f53ead PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:32.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c223a6cb-f933-4e8f-aba4-6c6c71d0e2c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:32.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c223a6cb-f933-4e8f-aba4-6c6c71d0e2c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f6f7f94 2021-05-28T01:13:32.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c223a6cb-f933-4e8f-aba4-6c6c71d0e2c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:32.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c223a6cb-f933-4e8f-aba4-6c6c71d0e2c2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:35.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464db81f-09c8-403d-acc7-b48ea73a31de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:35.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464db81f-09c8-403d-acc7-b48ea73a31de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@782de346 2021-05-28T01:13:35.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464db81f-09c8-403d-acc7-b48ea73a31de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:35.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=464db81f-09c8-403d-acc7-b48ea73a31de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:13:35.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:13:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:13:37.164Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:13:37.164Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:13:37.724Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc137065-15d1-4794-a879-d0c49a10ef03 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:37.724Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc137065-15d1-4794-a879-d0c49a10ef03 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fed988d 2021-05-28T01:13:37.724Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc137065-15d1-4794-a879-d0c49a10ef03 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:37.724Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc137065-15d1-4794-a879-d0c49a10ef03 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437856f6-7882-4144-aba1-5ef26fc33ca8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437856f6-7882-4144-aba1-5ef26fc33ca8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e5ad400 2021-05-28T01:13:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437856f6-7882-4144-aba1-5ef26fc33ca8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437856f6-7882-4144-aba1-5ef26fc33ca8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=189a99a2-ecaa-4c7d-b4b1-ef792dafed8e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=189a99a2-ecaa-4c7d-b4b1-ef792dafed8e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3acd9f80 2021-05-28T01:13:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=189a99a2-ecaa-4c7d-b4b1-ef792dafed8e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:44.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=189a99a2-ecaa-4c7d-b4b1-ef792dafed8e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:46.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5558ec3f-f0f8-4b49-a2b9-0c4794738058 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:46.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5558ec3f-f0f8-4b49-a2b9-0c4794738058 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ded7c4c 2021-05-28T01:13:46.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5558ec3f-f0f8-4b49-a2b9-0c4794738058 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:46.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5558ec3f-f0f8-4b49-a2b9-0c4794738058 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:50.033Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1237e69a-dca4-43ab-974b-ebe40bc93de0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:50.033Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1237e69a-dca4-43ab-974b-ebe40bc93de0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ee7a08c 2021-05-28T01:13:50.033Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1237e69a-dca4-43ab-974b-ebe40bc93de0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:50.033Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1237e69a-dca4-43ab-974b-ebe40bc93de0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:13:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:13:52.115Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:13:52.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=639e5d41-92a6-4927-98e1-c23399e2b78b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:52.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=639e5d41-92a6-4927-98e1-c23399e2b78b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66f58f84 2021-05-28T01:13:52.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=639e5d41-92a6-4927-98e1-c23399e2b78b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:52.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=639e5d41-92a6-4927-98e1-c23399e2b78b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4035cc80-d29c-4487-9547-1c30f8c30698 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4035cc80-d29c-4487-9547-1c30f8c30698 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30923d45 2021-05-28T01:13:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4035cc80-d29c-4487-9547-1c30f8c30698 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4035cc80-d29c-4487-9547-1c30f8c30698 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:13:58.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ec569e-6fac-434f-a0b6-f0fdc637924e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:13:58.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ec569e-6fac-434f-a0b6-f0fdc637924e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76db3807 2021-05-28T01:13:58.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ec569e-6fac-434f-a0b6-f0fdc637924e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:13:58.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ec569e-6fac-434f-a0b6-f0fdc637924e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:01.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c69754f0-30cd-41bf-8a10-324dd8f4cd0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:01.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c69754f0-30cd-41bf-8a10-324dd8f4cd0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@450b0590 2021-05-28T01:14:01.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c69754f0-30cd-41bf-8a10-324dd8f4cd0d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:01.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c69754f0-30cd-41bf-8a10-324dd8f4cd0d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:04.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6333656b-d22a-43bc-b545-1bf703c04a70 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:04.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6333656b-d22a-43bc-b545-1bf703c04a70 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fc83b1a 2021-05-28T01:14:04.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6333656b-d22a-43bc-b545-1bf703c04a70 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:04.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6333656b-d22a-43bc-b545-1bf703c04a70 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:14:05.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:14:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c487767c-1ba4-4d90-be35-e75b09153738 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c487767c-1ba4-4d90-be35-e75b09153738 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e9147b3 2021-05-28T01:14:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c487767c-1ba4-4d90-be35-e75b09153738 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c487767c-1ba4-4d90-be35-e75b09153738 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:10.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6311abc6-4754-4c5f-a4bd-d8ff1c3a5fa6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:10.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6311abc6-4754-4c5f-a4bd-d8ff1c3a5fa6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a54966 2021-05-28T01:14:10.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6311abc6-4754-4c5f-a4bd-d8ff1c3a5fa6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:10.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6311abc6-4754-4c5f-a4bd-d8ff1c3a5fa6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:13.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbaec4b8-752d-4897-98b4-106ec1d32478 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:13.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbaec4b8-752d-4897-98b4-106ec1d32478 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c1ffb47 2021-05-28T01:14:13.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbaec4b8-752d-4897-98b4-106ec1d32478 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:13.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbaec4b8-752d-4897-98b4-106ec1d32478 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:17.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b84725-16b4-4b09-93d6-ee24e6fc918d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:17.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b84725-16b4-4b09-93d6-ee24e6fc918d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52c10335 2021-05-28T01:14:17.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b84725-16b4-4b09-93d6-ee24e6fc918d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:17.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b84725-16b4-4b09-93d6-ee24e6fc918d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:20.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f8f7ce-1b01-4760-a664-6cad9e403cb1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:20.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f8f7ce-1b01-4760-a664-6cad9e403cb1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76496a01 2021-05-28T01:14:20.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f8f7ce-1b01-4760-a664-6cad9e403cb1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:20.185Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70f8f7ce-1b01-4760-a664-6cad9e403cb1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:14:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:14:24.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=546f66a2-be03-4f55-9a07-60444990381d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:24.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=546f66a2-be03-4f55-9a07-60444990381d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d0ce3d7 2021-05-28T01:14:24.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=546f66a2-be03-4f55-9a07-60444990381d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:24.084Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=546f66a2-be03-4f55-9a07-60444990381d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8854654c-375c-4ac7-9efc-e84396d4822f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8854654c-375c-4ac7-9efc-e84396d4822f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e2aeef2 2021-05-28T01:14:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8854654c-375c-4ac7-9efc-e84396d4822f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:26.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8854654c-375c-4ac7-9efc-e84396d4822f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:30.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1107f3-e03a-4b33-b895-299f9550e21f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:30.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1107f3-e03a-4b33-b895-299f9550e21f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64288aa9 2021-05-28T01:14:30.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1107f3-e03a-4b33-b895-299f9550e21f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:30.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa1107f3-e03a-4b33-b895-299f9550e21f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:31.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8fb1ce-11ce-4fa7-8aff-f7cf16599bb6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:31.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8fb1ce-11ce-4fa7-8aff-f7cf16599bb6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@598cfd4f 2021-05-28T01:14:31.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8fb1ce-11ce-4fa7-8aff-f7cf16599bb6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:31.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8fb1ce-11ce-4fa7-8aff-f7cf16599bb6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:35.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b12f022f-1df0-42e5-87f0-0f60c0a927f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:35.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b12f022f-1df0-42e5-87f0-0f60c0a927f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63a4aff2 2021-05-28T01:14:35.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b12f022f-1df0-42e5-87f0-0f60c0a927f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:35.164Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b12f022f-1df0-42e5-87f0-0f60c0a927f8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:14:35.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:14:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:14:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:14:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:14:38.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e49bbd-1057-4e11-8945-a95bd3428a66 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:38.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e49bbd-1057-4e11-8945-a95bd3428a66 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a10900a 2021-05-28T01:14:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e49bbd-1057-4e11-8945-a95bd3428a66 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:38.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e49bbd-1057-4e11-8945-a95bd3428a66 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:40.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63312143-8337-4e01-a805-1cb80912d9b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:40.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63312143-8337-4e01-a805-1cb80912d9b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a4d2e23 2021-05-28T01:14:40.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63312143-8337-4e01-a805-1cb80912d9b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:40.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63312143-8337-4e01-a805-1cb80912d9b7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:44.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b584ec17-2d99-418d-a461-16639c01657f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:44.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b584ec17-2d99-418d-a461-16639c01657f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1495c8a0 2021-05-28T01:14:44.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b584ec17-2d99-418d-a461-16639c01657f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:44.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b584ec17-2d99-418d-a461-16639c01657f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:46.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9eda8b66-7cd2-462d-b78c-b1c5a6d1a4ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:46.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9eda8b66-7cd2-462d-b78c-b1c5a6d1a4ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52883eff 2021-05-28T01:14:46.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9eda8b66-7cd2-462d-b78c-b1c5a6d1a4ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:46.468Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9eda8b66-7cd2-462d-b78c-b1c5a6d1a4ad PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:49.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e8bfa66-ca4b-4380-ae8e-6064c306af2b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:49.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e8bfa66-ca4b-4380-ae8e-6064c306af2b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c966464 2021-05-28T01:14:49.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e8bfa66-ca4b-4380-ae8e-6064c306af2b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:49.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e8bfa66-ca4b-4380-ae8e-6064c306af2b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:14:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:14:52.118Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:14:52.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=013b69f9-b6a6-42ab-b111-f9936ae6f721 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:52.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=013b69f9-b6a6-42ab-b111-f9936ae6f721 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1fe975e3 2021-05-28T01:14:52.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=013b69f9-b6a6-42ab-b111-f9936ae6f721 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:52.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=013b69f9-b6a6-42ab-b111-f9936ae6f721 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:55.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f52408e-bb2c-4ce3-b20c-a51a2e3d609c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:55.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f52408e-bb2c-4ce3-b20c-a51a2e3d609c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25037e76 2021-05-28T01:14:55.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f52408e-bb2c-4ce3-b20c-a51a2e3d609c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:55.996Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f52408e-bb2c-4ce3-b20c-a51a2e3d609c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:14:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b072fe-6c1c-4365-9b0b-94d3c467882a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:14:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b072fe-6c1c-4365-9b0b-94d3c467882a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@366fd3ef 2021-05-28T01:14:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b072fe-6c1c-4365-9b0b-94d3c467882a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:14:58.781Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87b072fe-6c1c-4365-9b0b-94d3c467882a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:01.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4599c43-c03f-4520-8dd3-15c578906547 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:01.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4599c43-c03f-4520-8dd3-15c578906547 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59b53402 2021-05-28T01:15:01.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4599c43-c03f-4520-8dd3-15c578906547 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:01.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4599c43-c03f-4520-8dd3-15c578906547 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:15:05.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:15:05.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0386bb45-0a81-413c-a64b-649b8bb59431 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:05.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0386bb45-0a81-413c-a64b-649b8bb59431 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d2704ba 2021-05-28T01:15:05.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0386bb45-0a81-413c-a64b-649b8bb59431 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:05.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0386bb45-0a81-413c-a64b-649b8bb59431 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:07.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ed8527-9cce-48f4-8b05-33d5ec79ee57 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:07.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ed8527-9cce-48f4-8b05-33d5ec79ee57 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33693ba6 2021-05-28T01:15:07.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ed8527-9cce-48f4-8b05-33d5ec79ee57 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:07.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34ed8527-9cce-48f4-8b05-33d5ec79ee57 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:10.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b703cd23-b728-4d7b-9fd1-98054fe2cb2c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:10.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b703cd23-b728-4d7b-9fd1-98054fe2cb2c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b51768f 2021-05-28T01:15:10.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b703cd23-b728-4d7b-9fd1-98054fe2cb2c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:10.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b703cd23-b728-4d7b-9fd1-98054fe2cb2c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:14.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c889ad6-ff7a-41d0-b1b8-de6ba4f468db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:14.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c889ad6-ff7a-41d0-b1b8-de6ba4f468db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b3448c3 2021-05-28T01:15:14.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c889ad6-ff7a-41d0-b1b8-de6ba4f468db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:14.381Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c889ad6-ff7a-41d0-b1b8-de6ba4f468db PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:16.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7de1211c-fb0e-4c27-9d52-2d0fdffffc5b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:16.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7de1211c-fb0e-4c27-9d52-2d0fdffffc5b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e039fcb 2021-05-28T01:15:16.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7de1211c-fb0e-4c27-9d52-2d0fdffffc5b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:16.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7de1211c-fb0e-4c27-9d52-2d0fdffffc5b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@444d27a6 2021-05-28T01:15:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:19.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:20.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:15:20.513Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:15:22.874Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-05-28T01:15:22.875Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:119) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:61) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:81) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:55) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:258) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:163) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:446) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:111) ... 13 common frames omitted 2021-05-28T01:15:22.875Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:15:22.875Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b7ddee4a-0006-4c1c-ad9d-3445f11b8000 ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:15:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:15:26.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:15:27.884Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.76.215:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.76.215:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:15:27.960Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.170.166:9042 com.datastax.driver.core.exceptions.AuthenticationException: Authentication error on host cassandra.onap/10.242.170.166:9042: Error during authentication of user asdc_user : org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses. at com.datastax.driver.core.Connection$9.apply(Connection.java:518) at com.datastax.driver.core.Connection$9.apply(Connection.java:480) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:213) at com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:202) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:118) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1509) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-05-28T01:15:29.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:15:31.677Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:15:32.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae55253-d3ef-40e5-846b-87593eaf1ad2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:32.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae55253-d3ef-40e5-846b-87593eaf1ad2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3458d4ae 2021-05-28T01:15:32.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae55253-d3ef-40e5-846b-87593eaf1ad2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:32.081Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae55253-d3ef-40e5-846b-87593eaf1ad2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:32.484Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=eae55253-d3ef-40e5-846b-87593eaf1ad2 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:15:32.484Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=eae55253-d3ef-40e5-846b-87593eaf1ad2 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:15:33.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1b2731-a51d-4958-a9ee-51e1fea86471 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:33.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1b2731-a51d-4958-a9ee-51e1fea86471 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f5cd90a 2021-05-28T01:15:33.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1b2731-a51d-4958-a9ee-51e1fea86471 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:33.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1b2731-a51d-4958-a9ee-51e1fea86471 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:34.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29c928c4-daac-46a3-ae53-71f88e40151c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:34.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29c928c4-daac-46a3-ae53-71f88e40151c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1281a950 2021-05-28T01:15:34.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29c928c4-daac-46a3-ae53-71f88e40151c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:34.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29c928c4-daac-46a3-ae53-71f88e40151c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:15:35.510Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:15:36.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c148d797-04c1-48b5-a66a-377b023ab638 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:36.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c148d797-04c1-48b5-a66a-377b023ab638 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@608ef20a 2021-05-28T01:15:36.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c148d797-04c1-48b5-a66a-377b023ab638 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:36.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c148d797-04c1-48b5-a66a-377b023ab638 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:37.162Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:15:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:15:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:15:37.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2886c8f7-c482-4acd-8791-a6f9357a61f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:37.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2886c8f7-c482-4acd-8791-a6f9357a61f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@620aa8ed 2021-05-28T01:15:37.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2886c8f7-c482-4acd-8791-a6f9357a61f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:37.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2886c8f7-c482-4acd-8791-a6f9357a61f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:38.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c00c4-afcd-4d74-a591-056e561ae4bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:38.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c00c4-afcd-4d74-a591-056e561ae4bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bc7b92f 2021-05-28T01:15:38.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c00c4-afcd-4d74-a591-056e561ae4bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:38.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c00c4-afcd-4d74-a591-056e561ae4bc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:41.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c220e02-402f-4da2-a031-5411741796f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:41.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c220e02-402f-4da2-a031-5411741796f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24135c2e 2021-05-28T01:15:41.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c220e02-402f-4da2-a031-5411741796f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:41.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c220e02-402f-4da2-a031-5411741796f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:43.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a63e17aa-770d-4a4d-95e3-b351672185e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:43.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a63e17aa-770d-4a4d-95e3-b351672185e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@339ae404 2021-05-28T01:15:43.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a63e17aa-770d-4a4d-95e3-b351672185e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:43.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a63e17aa-770d-4a4d-95e3-b351672185e5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:46.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48363462-a1b7-4224-b8f9-03e878d23db1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:46.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48363462-a1b7-4224-b8f9-03e878d23db1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e07f5a 2021-05-28T01:15:46.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48363462-a1b7-4224-b8f9-03e878d23db1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:46.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48363462-a1b7-4224-b8f9-03e878d23db1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:50.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:15:50.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:15:52.129Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:15:53.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c88f191-0fda-452c-ba5e-84894ebfe269 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:53.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c88f191-0fda-452c-ba5e-84894ebfe269 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e8a028 2021-05-28T01:15:53.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c88f191-0fda-452c-ba5e-84894ebfe269 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:53.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c88f191-0fda-452c-ba5e-84894ebfe269 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:54.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0bfa029-22ed-40cc-91c7-c01f8f60e935 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:54.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0bfa029-22ed-40cc-91c7-c01f8f60e935 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ca46e77 2021-05-28T01:15:54.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0bfa029-22ed-40cc-91c7-c01f8f60e935 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:54.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0bfa029-22ed-40cc-91c7-c01f8f60e935 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:56.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2896d021-5bcc-4b80-91f5-4e9eaf1c3f38 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:56.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2896d021-5bcc-4b80-91f5-4e9eaf1c3f38 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@587a1723 2021-05-28T01:15:56.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2896d021-5bcc-4b80-91f5-4e9eaf1c3f38 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:56.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2896d021-5bcc-4b80-91f5-4e9eaf1c3f38 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:15:58.904Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5bdbdcf-a3d6-4fe5-ad6e-b6c2182a7a36 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:15:58.904Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5bdbdcf-a3d6-4fe5-ad6e-b6c2182a7a36 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5fe7a9d9 2021-05-28T01:15:58.904Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5bdbdcf-a3d6-4fe5-ad6e-b6c2182a7a36 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:15:58.904Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5bdbdcf-a3d6-4fe5-ad6e-b6c2182a7a36 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:02.178Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f381bf3a-2fdb-43b7-9c5c-6bf37265bff0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:02.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f381bf3a-2fdb-43b7-9c5c-6bf37265bff0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18c9bbed 2021-05-28T01:16:02.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f381bf3a-2fdb-43b7-9c5c-6bf37265bff0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:02.179Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f381bf3a-2fdb-43b7-9c5c-6bf37265bff0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:05.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:16:05.514Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:16:05.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6adc1b72-77c2-4b53-9f50-8d40e5125749 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:05.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6adc1b72-77c2-4b53-9f50-8d40e5125749 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@701b955 2021-05-28T01:16:05.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6adc1b72-77c2-4b53-9f50-8d40e5125749 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:05.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6adc1b72-77c2-4b53-9f50-8d40e5125749 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:08.385Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c1c7419-d918-403c-9d5e-b5b1cff0b419 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:08.385Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c1c7419-d918-403c-9d5e-b5b1cff0b419 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f4a7599 2021-05-28T01:16:08.385Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c1c7419-d918-403c-9d5e-b5b1cff0b419 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:08.385Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c1c7419-d918-403c-9d5e-b5b1cff0b419 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45883958-c3b2-4685-8289-0cf44c8dd6ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45883958-c3b2-4685-8289-0cf44c8dd6ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d01acc8 2021-05-28T01:16:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45883958-c3b2-4685-8289-0cf44c8dd6ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:10.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45883958-c3b2-4685-8289-0cf44c8dd6ec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:14.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d782723e-9d54-4f77-a647-607292b4463b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:14.071Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d782723e-9d54-4f77-a647-607292b4463b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38426456 2021-05-28T01:16:14.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d782723e-9d54-4f77-a647-607292b4463b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:14.072Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d782723e-9d54-4f77-a647-607292b4463b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:17.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=155345e2-793d-4edb-85b2-7b19f6428a19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:17.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=155345e2-793d-4edb-85b2-7b19f6428a19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3283b91a 2021-05-28T01:16:17.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=155345e2-793d-4edb-85b2-7b19f6428a19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:17.277Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=155345e2-793d-4edb-85b2-7b19f6428a19 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:20.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:16:20.512Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:16:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:16:23.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:16:24.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e4685b-b5b3-42be-b5b2-7030ef9847fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:24.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e4685b-b5b3-42be-b5b2-7030ef9847fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6eb9e46e 2021-05-28T01:16:24.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e4685b-b5b3-42be-b5b2-7030ef9847fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:24.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16e4685b-b5b3-42be-b5b2-7030ef9847fd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:25.975Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:16:26.296Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8efa4f9f-e62c-48ee-a270-9909f150c562 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:26.297Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8efa4f9f-e62c-48ee-a270-9909f150c562 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ebe729d 2021-05-28T01:16:26.297Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8efa4f9f-e62c-48ee-a270-9909f150c562 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:26.297Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8efa4f9f-e62c-48ee-a270-9909f150c562 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:26.470Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=8efa4f9f-e62c-48ee-a270-9909f150c562 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:16:26.470Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=8efa4f9f-e62c-48ee-a270-9909f150c562 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:16:27.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68eeaee-fd81-4258-a8ad-a45f968fe3c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:27.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68eeaee-fd81-4258-a8ad-a45f968fe3c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25f5d7c8 2021-05-28T01:16:27.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68eeaee-fd81-4258-a8ad-a45f968fe3c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:27.374Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68eeaee-fd81-4258-a8ad-a45f968fe3c3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:28.573Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b68eeaee-fd81-4258-a8ad-a45f968fe3c3 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:16:28.573Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=b68eeaee-fd81-4258-a8ad-a45f968fe3c3 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:16:29.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5519d38e-52b2-4798-8edd-66ed74201ced PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:29.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5519d38e-52b2-4798-8edd-66ed74201ced PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a91e105 2021-05-28T01:16:29.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5519d38e-52b2-4798-8edd-66ed74201ced PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:29.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5519d38e-52b2-4798-8edd-66ed74201ced PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:31.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408d95b0-7631-4a2d-a237-b72806d10b53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:31.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408d95b0-7631-4a2d-a237-b72806d10b53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b2afead 2021-05-28T01:16:31.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408d95b0-7631-4a2d-a237-b72806d10b53 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:31.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408d95b0-7631-4a2d-a237-b72806d10b53 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:35.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df63751a-60a8-4489-ab23-5c95b435da25 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:35.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df63751a-60a8-4489-ab23-5c95b435da25 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@567d5bcf 2021-05-28T01:16:35.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df63751a-60a8-4489-ab23-5c95b435da25 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:35.087Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df63751a-60a8-4489-ab23-5c95b435da25 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:35.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:16:35.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:16:37.163Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-05-28T01:16:37.163Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-05-28T01:16:37.163Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-05-28T01:16:37.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d2bc64-2495-406e-acac-9cd15c7c8a4f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:37.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d2bc64-2495-406e-acac-9cd15c7c8a4f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d3d9d5f 2021-05-28T01:16:37.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d2bc64-2495-406e-acac-9cd15c7c8a4f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:37.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d2bc64-2495-406e-acac-9cd15c7c8a4f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78159190-d0db-4896-a812-0e47e53036fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78159190-d0db-4896-a812-0e47e53036fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58a1d634 2021-05-28T01:16:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78159190-d0db-4896-a812-0e47e53036fc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:40.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78159190-d0db-4896-a812-0e47e53036fc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:45.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c56a10-1b6c-45ab-b05f-4a23a6efb735 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:45.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c56a10-1b6c-45ab-b05f-4a23a6efb735 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2482841b 2021-05-28T01:16:45.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c56a10-1b6c-45ab-b05f-4a23a6efb735 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:45.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c56a10-1b6c-45ab-b05f-4a23a6efb735 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:46.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b73f0b-26ee-448a-aecc-3b75f99716e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:46.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b73f0b-26ee-448a-aecc-3b75f99716e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@181d0067 2021-05-28T01:16:46.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b73f0b-26ee-448a-aecc-3b75f99716e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:46.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04b73f0b-26ee-448a-aecc-3b75f99716e5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-05-28T01:16:50.469Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-05-28T01:16:50.506Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:16:50.516Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:16:51.116Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac4006b4-1230-499d-ba56-bfef874c055e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:51.117Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac4006b4-1230-499d-ba56-bfef874c055e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29fbb15e 2021-05-28T01:16:51.117Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac4006b4-1230-499d-ba56-bfef874c055e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:51.117Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac4006b4-1230-499d-ba56-bfef874c055e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:52.132Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:16:52.575Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.102.221 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-7697c4f957-9fbvk TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-05-28T01:16:53.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6f725d5-1ec2-49c1-b15d-cc4075bfb369 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:53.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6f725d5-1ec2-49c1-b15d-cc4075bfb369 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f3dd223 2021-05-28T01:16:53.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6f725d5-1ec2-49c1-b15d-cc4075bfb369 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:53.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6f725d5-1ec2-49c1-b15d-cc4075bfb369 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:54.159Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=e6f725d5-1ec2-49c1-b15d-cc4075bfb369 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:16:54.159Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=e6f725d5-1ec2-49c1-b15d-cc4075bfb369 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-05-28T01:16:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32278765-0c5c-4e23-9a68-541ee3c86fa0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32278765-0c5c-4e23-9a68-541ee3c86fa0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4faf3383 2021-05-28T01:16:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32278765-0c5c-4e23-9a68-541ee3c86fa0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:55.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32278765-0c5c-4e23-9a68-541ee3c86fa0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:16:56.000Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=32278765-0c5c-4e23-9a68-541ee3c86fa0 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:16:56.000Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=32278765-0c5c-4e23-9a68-541ee3c86fa0 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-05-28T01:16:58.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0749e0f4-6627-4a93-951c-3b8487030cbe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:16:58.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0749e0f4-6627-4a93-951c-3b8487030cbe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f410c29 2021-05-28T01:16:58.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0749e0f4-6627-4a93-951c-3b8487030cbe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:16:58.877Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0749e0f4-6627-4a93-951c-3b8487030cbe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:17:01.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88758b83-7060-463c-81e8-06f1cc698b04 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:17:01.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88758b83-7060-463c-81e8-06f1cc698b04 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4bc43bde 2021-05-28T01:17:01.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88758b83-7060-463c-81e8-06f1cc698b04 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:17:01.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88758b83-7060-463c-81e8-06f1cc698b04 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:17:04.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=214ab6cd-9566-4850-a312-33add48193ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:17:04.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=214ab6cd-9566-4850-a312-33add48193ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a79444f 2021-05-28T01:17:04.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=214ab6cd-9566-4850-a312-33add48193ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:17:04.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=214ab6cd-9566-4850-a312-33add48193ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:17:05.507Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-05-28T01:17:05.511Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-05-28T01:17:08.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a04bed-dd66-4271-9812-62d6de1732b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:17:08.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a04bed-dd66-4271-9812-62d6de1732b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cfc47a8 2021-05-28T01:17:08.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a04bed-dd66-4271-9812-62d6de1732b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:17:08.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a04bed-dd66-4271-9812-62d6de1732b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:17:10.881Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6894f11a-a340-48c5-86cc-a4fc8743f173 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:17:10.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6894f11a-a340-48c5-86cc-a4fc8743f173 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5beb2e5a 2021-05-28T01:17:10.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6894f11a-a340-48c5-86cc-a4fc8743f173 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:17:10.882Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6894f11a-a340-48c5-86cc-a4fc8743f173 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:17:13.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760ba88f-e58e-4a44-b8e1-f5e42ee86ede PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:17:13.980Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760ba88f-e58e-4a44-b8e1-f5e42ee86ede PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@311bf432 2021-05-28T01:17:13.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760ba88f-e58e-4a44-b8e1-f5e42ee86ede PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:17:13.981Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760ba88f-e58e-4a44-b8e1-f5e42ee86ede PartnerName=SDC.BE The cassandra down nodes number is 0 2021-05-28T01:17:16.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c20428e-a207-4a7b-9a40-2234d2973f9a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-05-28T01:17:16.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c20428e-a207-4a7b-9a40-2234d2973f9a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21508562 2021-05-28T01:17:16.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c20428e-a207-4a7b-9a40-2234d2973f9a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-05-28T01:17:16.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c20428e-a207-4a7b-9a40-2234d2973f9a PartnerName=SDC.BE The cassandra down nodes number is 0